Building a General Purpose Pedagogical Agent in a Web-Based Multimedia Clinical Simulation System for Medical Education
1. Knowledge model: This is a database that contains knowledge on specific topics which an expert in that particular domain would reasonably be expected to possess. In our system, the knowledge model contains data related to many different clinical cases which are to be learned.
2. Student model: This model provides a mechanism for assessing the state of the student's current knowledge of the information held within the knowledge base.
3. Tutor model: This model tries to play the role of a human tutor and is responsible for managing the overall learning environment.
1. The student may be surprised at the 30 percent of his/her correct answers in this example, realizing that he/she is not doing a good job in the current section, and would like to go back to this section to re-examine the questions more carefully without further hints. In this case, the student would select "No" and go back to the section to reselect more items. In our experiments, surprisingly, the students thought that it was a very useful aid for them to practice the case. It is particularly useful for a novice since it gives the student a rough feeling about how well he/she is doing. When the student finishes up the second trial for the section, the system will re-evaluate his/her performance again and the same process will repeat itself.
2. If the student selects "Yes," then the agent will ask the student whether or not he/she wants to get one correct answer at a time or get all of the correct answers at the same time, as shown in Figs. 4a and 4b, respectively. In the former case, the student can click the "view one more item at a time" button and one of the answer items will be shown first so that he/she can think about why the item should be selected first before more details about the item are revealed. For instance, for the example shown in Fig. 4b, the item RBC morphology is shown. When the student clicks on "View" in the Report column, through the hyperlink, the student can read more information pertaining to the item, such as why this item should be selected (the rationale of this item provided by the author) and what RBC morphology is about as shown in Fig. 4c. If the hyperlink "The rationale of this item provided by the author" in Fig. 4c is clicked, the rationale of the item in Fig. 4d is displayed. This arrangement will allow the student to think about how to approach the correct answer incrementally and give the student more opportunities to think about the problems instead of just reading the answer directly.
1. The instructor module arranges the teaching cases to be learned.
2. The student can set the hints level or simply ignore it (in this case, the default value is used).
3. Based on the teaching case selected by the student, the instructor module references the student profile database and the knowledge model database, then uses a rule-based engine to present the teaching case to the student in the PBL fashion described earlier.
4. The student's answers are stored in a log database.
5. At the end of each section, the agent evaluates the student's answers against the suggested items in the knowledge model, compares the evaluation result with the hints level, and offers hints to the student accordingly through the horizontal scaffolding engine, storing them in the student learning database for future usage.
6. The horizontal scaffolding engine also records the results in the horizontal scaffolding database.
7. At each checkpoint, the vertical scaffolding engine references the horizontal scaffolding database and the knowledge model, and provides the vertical scaffolding to the student. Although the architecture of the agent is not simple, there is no huge amount of computation involved. As a result, the students do not really feel any delay while interacting with the system with the existence of the pedagogical agent.
5.4.1 Analysis of the t-Test As described before, we measured how many times the students had tried before they got at least the 60 percent of the correct answers. The 60 percent score in group B does not include those answers provided by the agent. By using an Independent Samples t-test procedure, where is considered to be significant, the average, standard deviation, and P-value are computed by the software package SPSS 12.0 and are listed in Table 1. In Table 1, it can be found that there are significant differences between groups A and B in all the cases . Apparently, the pedagogical agent plays an important role in their learning activities.
5.4.2 Analysis of Questionnaire Table 2 uses a five-point Likert Scale and summarizes the students' responses to a series of questions posed by the current researchers. The students have also made a variety of comments after using the various teaching cases in the system. The principal results of this study may be summarized as follows (some of the results are provided by the instructors):
1. As expected, the students all felt that the teaching cases with the pedagogical agent were much more helpful than those without the agent, especially for their thinking process. The results of the t-test also demonstrated the significance of the pedagogical agent.
2. All the students agreed that the system significantly improves their skills in dealing with actual clinical cases as shown in Table 2. The standard deviation values are very small. This implies that there is a consensus regarding the students' responses to the agent. We have also interviewed the students who have experiences with the HINTS with and without the pedagogical agent. They expressed that without the agent, when they do not have any idea on how to work on a case and they do not get any hints from the system, they simply do not know what to do other than making random guesses. In other words, they are completely on their own. With the agent, they can get some hints from the system. Furthermore, they can control how many hints they want to get from the agent. For instance, for any given clinical case, there are more than 2,600 possible laboratory test items for them to select. When they call on the agent, the agent can tell them which items should be selected little by little under their control. The hints do provide some clues and thinking directions for them to think about the case. Of course, sometimes with the hints, they are still confused. In any case, they are certainly better off with the agent.
3. The instructors feel that although the agent does not handle the students' problems intelligently in the sense that it cannot answer individual or specific problems, it does save the instructors a lot of time in working with the students throughout the teaching cases. They believe that the agent can serve students as a teaching assistant in their learning process.
4. From the system logs of the students' learning, the average times the students in groups A and B spent on a case are 51 and 37 minutes, respectively. In the student interviews, the students had expressed that they would like to be able to spend about 40 minutes on average on a case. Apparently, the agent indeed can make the learning more efficient and enable students to finish a case within 40 minutes.
1. Although the agent does not really intelligently detect a student's misconceptions or provide them with customized hints, it can at least save the human instructors some time and trouble going through a teaching case with the students and providing constant guidance to the students.
2. Although a human instructor can certainly do a better job than the agent in terms of tutoring, the presence of a human instructor may put the students under some pressure. Using the agent, the students feel they can work through the teaching cases in a more relaxed and stress-free environment. In particular, the agent is indeed extremely useful for novices. This is because although they have enough basic medical knowledge for the clinical cases, they do not have enough experience in applying their knowledge to the cases. The agent is able to provide them with useful hints.
3. The teaching case needs to be developed by the domain experts without the development of the pedagogical agent in mind. However, the agent inhabiting the system can be easily developed without the "explicit involvement" of a domain expert since the agent simply reveals the items which need to be selected with their rationale and related resources little by little under the students' control. Of course, the hints information is originally provided by the domain experts as the answers to the problems under study instead of the hints to the students. In other words, the hints are provided by the domain experts implicitly and the system makes use of the agent mechanism to provide the hints information at the right time to the students when they need the hints.
4. Despite the preliminary success of the agent development, ultimately there is still a need to make the agent more intelligent in order to cope with the various and inevitable problems that students encounter during their learning process.
5. As discussed in Section 5.2, the pedagogical agent should not provide too many hints to the students without costing the students anything. Otherwise, the students will take the hints for granted without really thinking about how to solve the problems on their own and always try to rely on the hints instead of working with the system on their own.
6. Although the theoretical aspect of the pedagogical agents has been well-documented in literature, less efforts have been made on how a pedagogical agent should be implemented in a real computerized simulation learning environment. Here, we propose and implement a pedagogical agent architecture that inhabits the HINTS to further facilitate the students' learning and make the HINTS more efficient for learning. Our focus is the design of the pedagogical architecture and its implementation. A more in-depth evaluation of the effectiveness of the system is still needed in the future.
• Y.-M. Cheng is with the Department of Computer Science and Information Engineering, Shu Te University, No. 59, Hengshan Road, Yanchao, Kaohsiung County 824, Taiwan, ROC. E-mail: firstname.lastname@example.org.
• L.-S. Chen, S.-F. Weng, and Y.-G. Chen are with the Department of Electrical Engineering, National Cheng Kung University, No. 1, University Road, Tainan City 701, Taiwan, ROC.
E-mail: email@example.com, firstname.lastname@example.org, email@example.com.
• H.-C. Huang is with the Institute of Computer and Communication Engineering, National Cheng Kung University, No. 1, University Road, Tainan City 701, Taiwan, ROC. E-mail: firstname.lastname@example.org.
• C.-H. Lin is with the College of Medicine, National Cheng Kung University, No. 1, University Road, Tainan 701, Taiwan, ROC.
Manuscript received 13 May 2008; revised 16 Nov. 2008; accepted 1 Apr. 2009; published online 9 Apr. 2009.
For information on obtaining reprints of this article, please send e-mail to: email@example.com, and reference IEEECS Log Number TLT-2008-05-0038.
Digital Object Identifier no. 10.1109/TLT.2009.18.
Yuh-Ming Cheng received the PhD degree in electrical engineering from the National Cheng Kung University, Taiwan. Since February 2005, he has been an associate professor in computer science and information engineering at Shu Te University, Taiwan. His research interests include software engineering, multimedia Web application system, e-Learning, game-based learning, and sensor network applications. He is a reviewer for international journals such as the Journal of Educational Technology & Society and Computers & Education.
Lih-Shyang Chen received the PhD degree in computer science from the University of Pennsylvania in 1987. Since August 1990, he has been with the Department of Electrical Engineering at National Cheng Kung University, where he is currently a professor. His research interests include computer graphics, image processing, pattern recognition, network systems, multimedia Web application systems, and e-Learning systems.
Hui-Chung Huang is a student at the Institute of Computer and Communication Engineering, National Cheng Kung University, Taiwan. His main research focuses on the multimedia system and Web application.
Sheng-Feng Weng received the master's degree from the National Cheng Kung University, Taiwan. He is currently a doctoral student in the Department of Electrical Engineering, National Cheng Kung University, Taiwan. His main research focuses on the multimedia system, Web application, and medical education system.
Yong-Guo Chen received the MD degree in medicine from China Medical University, Taiwan, and the second degree in medical information science from Taipei Medical University. Since August 1994, he has been on deputation to the hospital for his resident physician practice. He is currently a doctoral student in the Department of Electrical Engineering, National Cheng Kung University, Taiwan. His main research focuses on medical informatics, medical multimedia, collaboration learning in medical education, electronic medical record, and medical AI.
Chyi-Her Lin is a professor of pediatrics and dean for medical education, College of Medicine, National Cheng Kung University. His research interests include medical education and neonatology.