Pages: pp. 130-142
Abstract—Multimedia educational resources play an important role in education, particularly for distance learning environments. With the rapid growth of the multimedia web, large numbers of educational video resources are increasingly being created by several different organizations. It is crucial to explore, share, reuse, and link these educational resources for better e-learning experiences. Most of the video resources are currently annotated in an isolated way, which means that they lack semantic connections. Thus, providing the facilities for annotating these video resources is highly demanded. These facilities create the semantic connections among video resources and allow their metadata to be understood globally. Adopting Linked Data technology, this paper introduces a video annotation and browser platform with two online tools: Annomation and SugarTube. Annomation enables users to semantically annotate video resources using vocabularies defined in the Linked Data cloud. SugarTube allows users to browse semantically linked educational video resources with enhanced web information from different online resources. In the prototype development, the platform uses existing video resources for the history courses from the Open University (United Kingdom). The result of the initial development demonstrates the benefits of applying Linked Data technology in the aspects of reusability, scalability, and extensibility.
Index Terms—Distance learning, e-learning, educational video resources, Semantic Web, linked data, semantic annotation, semantic search, web services.
In the modern world e-learning activities are essential for distance learning in higher education. More than 5 million students have used or are using at least one online course in their studies, and the number of online students is growing by 25 percent every year [ 1]. The digital video, as one type of the multimedia educational resource, plays a key role in distance learning environments [ 2]. With rapidly growing numbers of digital educational video resources being created, it is important to accurately describe the video content and enable the searching of potential videos in order to enhance the quality and features of e-learning systems [ 3].
The Open University (OU) is the leading university in the United Kingdom for providing e-learning courses and it serves around 200,000 students at all degree levels. In collaboration with the British Broadcasting Corporation (BBC), the OU has produced a wide range of television programs (e.g., documentaries, historical event, news, and scientific programs) that can serve both students and general audiences.
Different OU departments manage their own educational resources separately because the resources, especially video resources, are produced by different partners under heterogeneous licenses and constraints at different times. However, some resources are related to one another and can serve multiple courses. With the rapid growth of the multimedia web, a large number of free educational resources are also available on the web. Therefore, it is crucial to gain the capability to efficiently search for all related distributed educational resources together to allow them to be used to enhance the learning activities. To this end, this paper has identified the following primary challenges.
This paper adopts Semantic Web technology, more precisely, the Linked Data approach to address the above challenges. The following lists the major contributions of our approach.
The remainder of the paper covers background and related work discussions (Section 2), the overall platform architecture (Section 3), the detailed illustration about the annotation process and the Annomation tool (Section 4), the detailed description about the SugarTube browser (Section 5), lessons learned from the survey-based evaluation process (Section 6), and the conclusion and future work (Section 7).
Videos are important educational resources that enable students to gain knowledge more efficiently and intuitively than text-based educational resources. Video resources play an important role in distance learning courses (e.g., history courses). For example, a five-minute long video of a speech may contain plenty of information such as event background, location, time and related people. However, traditional educational video resources usually lack labeled vocabularies and structured metadata. These drawbacks limit the usability, efficiency, and reusability of the educational video resources.
To improve e-learning outcomes, educational video resources should have accurate and collaborative annotations generated by domain experts, course creators, and tutors. It is important that the annotation vocabularies are accurate, identifiable, and sharable between different groups of people. Furthermore, if each piece of the annotation in the videos is detailed with further information, this would help students to view a more complete picture of a learning topic. Moreover, if these annotations are linkable to other relevant learning data from both internal and external resources, then it would enable students to gain a more comprehensive understanding of the topic from different perspectives.
The Semantic Web [ 5] is an evolving development of the World Wide Web, in which the meanings of information on the web is defined; therefore, it is possible for machines to process it. The basic idea of Semantic Web is to use ontological concepts and vocabularies to accurately describe contents in a machine readable way. These concepts and vocabularies can then be shared and retrieved on the web. In the Semantic Web, each fragment of the description is a triple, based on Description Logic [ 6]. Thus, the implicit connections and semantics within the description fragments can be reasoned using Description Logic theory and ontological definitions. Earlier research work on the Semantic Web focused on defining domain specific ontologies and reasoning technologies. Therefore, data are only meaningful in certain domains and are not connected to each other from the World Wide Web point of view, which certainly limits the contributions of Semantic Web for sharing and retrieving contents within a distributed environment.
Linked Data [ 7] is the recent revolutionary development of the Semantic Web. Linked Data create typed links between different data from different resources. From the technical point of view, Linked Data means to publish data on the web in such a way that they are readable by machines and their meanings are explicitly expressed. These data are then linked to external data sets, and in turn are linked from external data sets [ 8]. Linked Data changes the way of organising knowledge-based resources on the web by using the following four principles [ 8]:
Linked Data can be easily queried through SQL-like languages (e.g., SPARQL [ 9]). The most promising data set of Linked Data is the Linked Open Data cloud [ 4] (see Fig. 1) that includes data in seven different areas such as media, geographic, publication, user-generated content, government, cross domain, and life science.
Figure Fig. 1. The Linked Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net.
Linked Data are published through web services in order to be accessed by various applications. In particular, resource-oriented RESTful web services [ 10] are naturally matched to the characteristics of publishing the Linked Data resources [ 11] into SPARQL endpoints.
In this paper, the services for retrieving Linked Data are defined as Linked Data Services in order to distinguish them from other services that do not deal with Linked Data.
The following summarizes four most important advantages of using Linked Data to create video annotations for the educational domain.
The prior work on video annotation tools can be summarized as either fat-client software rather than web browser based, or non-Linked Data annotations. An important early system was Vannotea [ 14] which relied on a dedicated client application to enable collaborative annotation, but the annotations were not in a Semantic Web style. M-OntoMat-Annotizer [ 15] did use Semantic Web annotations, linking them to annotations embedded using MPEG-7 [ 16]. However, MPEG-7 is initially designed based on nonsemantic XML description language. It focuses on video text, presentation models, pictures, graphics, audio volumes, and searching matrix with relating to information about the video [ 16]. Therefore, semantic enhancements of MPEG-7 are always fat clients. Other studies [ 17] use a domain ontology that describes the videos to classify annotations. However, the domain ontology-based annotations cannot annotate information from outside of their domain, and it is unlikely that students are interested in learning these in order to search videos.
Videos are traditionally searched by syntactic matching mechanisms (e.g., [ 18]). Recently, with more videos being annotated or tagged in the Linked Data manner, researchers have begun to search videos in a more Semantic-Web-oriented fashion. The two major approaches are the semantic indexing process and the natural language analysis process. The indexing process assumes that the video annotations are made from a fixed set of vocabularies that change infrequently (e.g., [ 19]). Although this process can be efficient, the fixed set of vocabulary may introduce a gap between user's knowledge and indexed annotations, especially in the educational environment, in which videos are often annotated by different groups of teachers or students, who may apply different annotation terms to the same video in the context of different courses and key points. The natural language analysis process focuses more on adding semantic tags to the user's search inputs (e.g., [ 20]). However, most of these approaches require machine-learning mechanisms to assist dynamically adding tags. Hence, they restrict their applications to small and closed domains of discourse.
Our approach adopts the principles of Linked Data to annotate the existing OU educational video resources and link them to other relevant resources on the web. Fig. 2 shows the overall architecture of the annotation and browser system, namely Annomation $+$ SugarTube.
Figure Fig. 2. The overall architecture of Annomation and SugarTube.
There are four groups of users:
Annomation is a collaborative Linked Data-based annotation platform that allows domain experts, course creators, and tutors to annotate videos using vocabularies from the Linked Open Data Cloud for different types of information (e.g., GeoName vocabularies for locational annotations). As mentioned earlier, the usage of Linked Data makes annotations accurate, distinguishable, and deferencable. Furthermore, these annotations are published as Linked Data. Therefore, they are linked to other related external educational resources annotated by semantically related Linked Data vocabularies.
SugarTube provides an online browsing platform that allows tutors and students to browse and search videos that are annotated by Annomation. It offers both syntactic and semantic search functionalities. The semantic search API not only finds video from the OU video repository but also delivers any linked educational resources from the Linked Open Data cloud to the user interface. The syntactic search offers more syntax-based related educational resources from the web.
The details of Annomation and SugarTube are discussed in Sections 4 and 5, respectively.
Annomation 2 is a web application that allows users to view a video in a collaborative way, pause it, and add Linked Data annotations to instants or durations on the video timeline.
The video annotation ontology and annotation instances are stored in a Sesame RDF [ 21] quad store, and the ontology reuses a number of RDF vocabularies. These vocabularies include:
To tie together the data described using these vocabularies, we designed a small vocabulary specific to the annotations, the core of which is:
A simplified high-level ontology explanation is that each annotation is created by a user for an instant or time duration on a video. An annotation example that ignores the namespace is listed below:
Traditional video annotations using free-text keywords or predefined vocabularies are insufficient for a collaborative and multilingual environment. They do not properly handle the annotation issues, such as accuracy, disambiguation, completeness, and multilinguality. For example, free-text keywords annotation easily fails on accuracy issues as they may contain spelling errors or be ambiguous. Furthermore, they are insufficient for a collaborative and multilingual environment.
Our approach uses Linked Data to tackle the above issues in video annotations. It brings the following benefits.
Five Linked Data Services are currently used as a foundation to annotate videos, and they are embedded in the Annomation functions to facilitate the annotation process. More services can be easily added into the system by adding a tab option to show the query results of the new service when required. The five Linked Data Services are:
The Annomation interface (see Fig. 3) is divided into four sections: a Flash video player (top left); a list of current annotations (top right); controls for the video player, and for entering new annotations (across the centre) and a set of panels to help the user to find new Linked Data URLs (bottom). The bottom panels provide quick access to previously used tags, to the Dewey and Library of Congress classifications schemes, Open University course taxonomies, a service for suggesting URLs based on the Zemanta service, and a visual map tool that uses GeoNames to find named geographical entities.
Figure Fig. 3. A snapshot of the Annomation interface.
SugarTube 10 is developed to facilitate the usage of the OU's educational video resources that are annotated by Annomation. It adopts the Semantic Web approach to search videos and explore their related online resources in a mashup navigation interface. In SugarTube, the annotations are semantically matched to other annotated educational resources from the web.
The SugarTube application includes three layers. Users interact with the application layer when specifying the concepts, documents (e.g., lecture notes), or website contents in order to get educational video resources. Based on different types of concept data, user requests are then sent to the semantic data mining and reasoning layer for generating different queries to the service layer. The service layer includes both Linked Data Services and nonsemantic-based services.
In addition to the Linked Data Services that are applied in the Annomation process, some other Linked Data Services and nonsemantic services are used in SugarTube. The Linked Data Services are:
The nonsemantic services are:
The SugarTube functionalities are divided into two groups, namely basic concept search and advanced search.
The basic concept search divides the concepts into “Person,” “Event,” “Place,” and “Others.” For different types of concepts, different service queries are generated. For instance, searching by the name of a person queries the searchByPerson WorldHistory service, while searching by the name of a place queries the searchByName GeoName service.
The advanced search supports searching videos by automatically analyzing documents, highlighting web contents, and pointing to locations on a map. Behind this, the Zemanta 17 service is used. For example, when a user copies and pastes the learning content from lecture notes into the textfield, all related knowledge concepts are listed, which enables the user to select further video searching activities. The Google map service is deployed for gathering the geoinformation about a place so that the user may click on the map to search related videos. The searching results do not only contain the OU educational video resources with their annotations but also include relevant learning resources about the videos and related videos from other services.
There are four different types of mining and reasoning processes: namely syntax parsing, document analysis, geographic mapping, and annotation inferencing.
The syntax parsing is the basic reasoning process to match syntax-based keywords to a URI identifier from the Linked Open Data Cloud. The syntax parsing process is triggered by the basic concept search functionality. The GeoName service is used for place syntax parsing and the WorldHistory service is used for event and person syntax searching. For instance, when a place name of “Cape Canaveral” is given as a searching keyword, the GeoName RDF service is allocated to search for the “Cape Canaveral” string. The result of the parsing is a RDF description (see Fig. 4) including the URI identifier ( http://www.geonames.org/4149910), the geographical information (latitude 28.45861 and longitude 80.5331), country (US), and different language spellings. These syntax parsing results are the fundamental elements which perform the further video repository query and advanced reasoning. For example, the URI identifier can be used to query videos annotated by the same URI (see Fig. 5). In the syntax parsing process, if there is more than one RDF instance found for the same syntax concept, a suggestion dialog box appears to allow users to specify which one is the target concept (e.g., Birmingham can refer to a city located in either the United Kingdom or United States).
Fig. 4. Syntax parsing results.
Fig. 5. SugarTube: video search results.
The document analysis process is used to analyze a document that is used to guide the study topic (e.g., the “Berlin Wall” historical topic). Typical documents are lecture notes and online webpages (including online slides). Currently, the Zemanta service is used for documentation analysis task. The analysis results are key learning points, knowledge, and concepts with their URI identifiers from the Linked Open Data cloud. For example (see Fig. 6), when a document about the “Berlin Wall” is processed, the key learning points in the document are identified as Berlin, Berlin Wall, Germany, East Germany and so on. These key points are matched to URI identifiers in DBpedia, Wikipedia, and Freebase for gaining further related educational resources.
Fig. 6. SugarTube: document analyzing results.
The geographic mapping process uses the Google map API to give students a geographical image to allow them to better understand the learning topic. The reasoning includes using the map information as the starting point to search for videos and other related learning resources, as well as parsing syntax or document analysis results to get the map. Taking the previous “Cape Canaveral” example, the latitude and longitude, which are gained by parsing the RDF results, are used to locate “Cape Canaveral” on the Google map (see Fig. 7).
Fig. 7. SugarTube: geographic reasoning result.
The annotation inferencing process uses the tree-structure advantages of the ontology-based semantic annotations. The annotation class definition has the properties of rdfs:subClassOf, owl:sameAs, and rdfs:seeAlso. For example, if http://dbpedia.org/page/UnitedKingdom, owl:sameAs http://www.freebase.com/view/en/unitedkingdom, then any videos annotated with either of these two URIs will be related to the other. By using the annotation reasoning process, the searching results are more accurate and widely covered. Although different video resource providers may use different Linked Data vocabularies to annotate their videos, they are linked together as search results through the SugarTube browser (see Fig. 8).
Fig. 8. Linked videos from different educational resources.
Fig. 9 displays the mashup results for a video search request. It consists of four main sections:
Currently, Annomation and SugarTube serve as the testing prototypes to the OU's history course teams and their students. Because they focus on two different user groups, an overall evaluation rate for the whole platform is determined by aggregating the two separate evaluations. We organized two evaluation sessions with different user groups, namely Experts and Tutors Evaluation Group (ETEG, 15 evaluators) and Students Evaluation Group (SEG 25 evaluators). The ETEG focuses on Annomation evaluation and the SEG focuses on SugarTube evaluation.
Figure Fig. 9. Data Mashup in the SugarTube application.
The members of ETEG all use eLearning systems on a daily basis for teaching distant learning students. In addition, three of them have experience of using textual tools for annotating the video learning materials. The SEG includes 18 female and seven male students from ages 18-28 who are studying part-time undergraduate history courses at first year level. None of the students had any experience of using professional educational video searching tools before but often use Google or Yahoo when searching online.
The evaluation process includes four steps:
The Annomation evaluation contains five tasks:
After the evaluation, we provided an Annomation evaluation questionnaire which consists of a rating for interface simplicity and usability, a rating for the quality and accuracy, identifying the most used annotation resources, identifying the most used annotation terms, and comments on using Linked Data technologies.
Fig. 10 shows that 11/15 users thought the Annomation interface is very simple to use and 10/15 users believe it is easy to find the correct annotations to use, while the questionnaire shows that users prefer to use Wikipedia or DBpedia as annotations, which is not surprising due to the high recognition rate of Wikipedia. However, the OU course classification vocabularies are used surprisingly rarely.
Figure Fig. 10. Annomation evaluation results of simplicity, usability, and most used annotation resources.
The most interesting elements to be annotated are person (12 votes), place (11 votes), and event (11 votes) that exactly match the basic concept search functions provided by the SugarTube. The most common view from the ETEG evaluation group (14/15 users agreed) is that the Linked Data-based annotations are much more accurate and explicit than other free-text-based annotations and much more scalable than domain ontology-based annotations.
Another important evaluation aspect is the performance. It is mainly evaluated by analyzing the time spent on completing the evaluation tasks. The analysis shows that most users can complete the simple annotation tasks in under 3 minutes and complex tasks within a 15-minute limit. There was only one failure report about the conversation mood annotation task (lacking knowledge of the topic).
By asking for comments regarding the answers to the questionnaire, the two major lessons have been learned:
Whether students think the SugarTube can help their studies is the most interesting part of our evaluation task for SEG. The SugarTube evaluation tasks contains:
The first chart in the Fig. 11 shows that 23/25 students believe SugarTube is “very helpful” or “helpful.” Only 2/25 students voted for “a little helpful” and no student thought it is not helpful at all. This is an encouraging message for us.
Figure Fig. 11. SugarTube evaluation results of helpfulness and time spending on the lecture notes search task.
By monitoring how long it takes the students to identify all the related videos and useful information for a history lecture note (500 words), we found that most of the students can finish this task within 10 minutes (see second chart of Fig. 11) by choosing to use the document analysis function. Consequently, students unanimously agree that the document analysis is the most useful function for their course preparations (basic concept search function is second, see the first chart in Fig. 12).
Figure Fig. 12. SugarTube evaluation results for most useful functionality and data.
The videos or data voted most useful come from the OU linked open data set, Openlearn, and Wikipedia/DBpedia (surprisingly, YouTube resources and TV resources are the most unpopular data, see the second chart in Fig. 12).
The most important lesson learned from the evaluation at this stage is that students are more interested in the data that comes directly from the education-oriented services rather than social information websites such as YouTube.
The other parts of SugarTube evaluation questionnaire consist of the rating of the usability, quality, and accuracy of the tool. 20/25 students voted the usability as “very good,” 22/25 students voted the quality as “very good,” and 24/25 students voted the accuracy as “very good.” The major concern is the response time of some searches at runtime. As the SugarTube is a search tool that invokes different Linked Data services at the same time after search request is received, services' response time are different because of the quality of their own services and servers' runtime workload. This is a tradeoff between the quality and the accuracy. Since we invoke different Linked Data services at the runtime, the newest information and various data are found, which reflects the high accuracy satisfaction rate in the survey.
Note that since both tools are still in prototype testing, there is still much work to be done to integrate them to the current OU distance learning systems and processes. The limitation of our evaluation is that we cannot evaluate how much the SugarTube can improve the tuition without applying it on live course teaching and examination processes.
This paper illustrated the Annomation and the SugarTube platform that uses Linked Data technologies to semantically annotate and search educational video resources from the Open University video repository and link the videos to other educational resources on the web.
In the semantic annotation process, 1) an annotation ontology is defined to support Linked Data annotations; 2) dynamic annotation URI suggestions are fully supported by integrating Linked Data Services into the Annomation interface; and 3) collaborative functionalities are implemented to enhance the teamwork capability.
In the semantic search process, the search methods are based on the data retrieved through Linked Data Services and URIs, which links different resources together to enrich the original video search results. SugarTube shows that e-learning resources distributed across different educational organizations can be linked together to provide more value-added information.
The contributions of introducing Linked Data technologies to annotate and browse multimedia educational resources are summarized as follows:
Further research work will integrate a context-aware annotation suggestion technique [ 32] into the Annomation application to speed up the annotation process. Furthermore, it is worth adding more Web 2.0 functionalities to the SugarTube browser to support better educational resources sharing between users. It is also important to integrate Semantic Web Service technologies, such as dynamic service discovery, invocation, and orchestration, to the applications for better usage of a wider range of the available Linked Data Services.
This research was partly funded by the European Commission's 7th Framework Program SOA4ALL project, NoTube project, and the Open University's Semantic Media Group.