The Community for Technology Leaders

Mood Recognition during Online Self-Assessment Tests

Christos N. Moridis
Anastasios A. Economides

Pages: pp. 50-61

Abstract—Individual emotions play a crucial role during any learning interaction. Identifying a student's emotional state and providing personalized feedback, based on integrated pedagogical models, has been considered to be one of the main limits of traditional tools of e-learning. This paper presents an empirical study that illustrates how learner mood may be predicted during online self-assessment tests. Here, a previous method of determining student mood has been refined based on the assumption that the influence on learner mood of questions already answered declines in relation to their distance from the current question. Moreover, this paper sets out to indicate that "exponential logic" may help produce more efficient models if integrated adequately with affective modeling. The results show that these assumptions may prove useful to future research.

Index Terms—Computer uses in education, education, human-centered computing, human information processing, personalization, self-assessment, Web-based interaction.


Contemporary computerized learning environments, whether Web-based or not, usually include a combination of carefully structured hypertext, animations, and test-based feedback [ 1], [ 2] in a well-organized and sound environment. In addition, an important emerging strand of current research aims to provide these systems with the ability to recognize a learner's emotional state and activate an appropriately tailored response based on integrated pedagogical models.

Experienced teachers can modify their teaching style according to the stimuli that they receive from their students, thus providing personalized attention. However, platforms for e-learning are incapable of receiving feedback from students. As a result, they can become inadequate for learning. Each student may or may not need additional or reduced support. Successful instructors have the ability to intuitively manage key human factors (such as passion, happiness, dislike, fear, will, frustration, satisfaction, etc.) to promote learning. Although the presence of technology is widespread in Web-based learning environments, it is not responsive to affective reactions experienced while using such learning environments.

Nowadays, the Web can integrate adequate technology and environment, where learners can be uniquely identified, content can be purposely presented, and progress can be individually monitored, supported, and assessed. Nevertheless, a theoretical comprehension of how individuals learn online is as yet absent [ 3]. In general, cognitive and affective solutions originally designed for the classroom only insufficiently meet the individual needs of Web learners.

This is what is defined as the "Plausibility Problem" [ 4, p. 102]: "Even if the computer could accurately diagnose the student's affective state and even if it could respond to that state (in combination with its diagnosis of the learner's cognitive state) exactly as a human tutor would, there remains one final potential difficulty: the plausibility, or perhaps the acceptability, problem. The issue here is whether the same actions and the same statements that human tutors use will have the same effect if delivered instead by a computer, even a computer with a virtually human voice."

For that reason, teaching practices and theories should be adequately adapted to tutoring systems and in a way convenient for developers to use. Accordingly, the method proposed in this paper is being originally developed and tested for online use aiming at personalizing the learning experience.

The recognition of a learner's emotional condition may play a vital role in ameliorating the effectiveness of e-learning [ 5]. Lack of emotion recognition has been considered to be one of the main limits of traditional tools of e-learning. This is an important issue, since student's performance during a learning session may be seriously hampered due to emotional reasons. When students are facing exams, this effect may be even more intense. Examination conditions require an integration of various skills: Students are expected to read, understand, analyze, apply their knowledge, and then present a structured answer to the questions [ 6]. However, these activities must be done within a limited time and often under strictly controlled conditions. As a result, students are often emotionally strained. Faced with sadness, worry, shame, frustration, or despair, people lose access to their own memory, reasoning, and the capacity to make mental associations [ 7].

The method being presented here could assist in developing a system that would help the student prepare cognitively and affectively before exams through online multiple choice question tests. Sometimes, it may be conducive to the learning experience to be self-assessed with the aid of a tutoring system, rather than with a real teacher. Research evidence has shown that people with more anxiety trust real humans less as their interaction partners [ 8]. So, such systems could be utilized by any kind of educator to configure online multiple choice question tests based on their course, so as to provide their students with the potential benefits of an affective tutoring system. Students could also use these systems after having studied each chapter of a book to self-assess their comprehension and progress in an affective way. In addition, any learner could benefit from the use of such systems depending on their learning goal, which can range from preparing oneself for a test such as the Graduate Record Examination (GRE) to self-assessing one's knowledge during an online course of any topic. A flexible e-learning system would take into consideration the student's current knowledge and learning preferences [ 9] to generate individualized learning paths [ 10]. In addition, the system would try to introduce students to an emotional state conducive to learning by providing adequate feedback [ 11].

The tutoring system affective routines will be embedded within learning and the effort to produce an adequate mood and an optimal emotional, motivational state for the current learning task will apply to the entire learning experience [ 12]. Thus, we are dealing with several different elements which need to be combined effectively to produce a new generation of tutoring systems. Therefore, a way to do this is to formulate and establish every constituent in separation and then try to determine how they can all be combined together to produce optimal results. This has to be a joint effort, bringing together scientists from various fields [ 13].

We have developed and evaluated a method in order to provide a measurement for the appraisal of student's mood with respect to each question the student is about to answer, and determine system feedback to the student [ 14]. This method has indicated an over 80 percent success rate in recognizing whether a student is in a negative or positive mood. In this paper, a better model is being introduced based on the assumption that student success or failure to the most recent questions influences their mood positively or negatively toward the current question. This has already been shown by the previous method, but now the model weights this effect according to how recent the correct or incorrect answer is. This means that the influence of a correctly or incorrectly answered question diminishes as the test proceeds and that the more recent a correct or incorrect answer is, the more heavily it influences student mood. In addition, the new method suggests that it may be beneficial to affective modeling to integrate some "exponential logic," since emotions may be better expressed logarithmically, in same way as senses. The new model was evaluated using the same set of data derived from the experiments conducted to check the previous method. The results confirmed the new supposition and led to a better method of evaluating student mood during an online self-assessment test.

Previous Work

The fundamental nature of affective factors in human cognitive procedures and learning has been acknowledged by numerous researchers [ 7], [ 15], [ 16], [ 17], [ 18], [ 19], [ 20], [ 21], [ 22], [ 23]. Consequently, during the last few decades, there have been attempts by several educators to develop learning strategies in order to take advantage of these issues [ 24], [ 25], [ 26], [ 27], [ 28]. In addition, various researchers have pointed to the need for developing tutoring systems with the ability to recognize a learner's emotional state and activate an appropriately tailored response based on integrated pedagogical models [ 5], [ 29], [ 30].

While some researchers address the issue of motivational skills concerning an intelligent tutoring system [ 29], [ 31], [ 32], others try to develop systems that focus on emotion [ 33], [ 34], [ 35]. However, to the authors' knowledge, there has been no previous effort to develop a tutoring system with mood regulation capacities. Motivation, emotion, and mood overlap, but have distinct characteristics as well. According to Bull et al. [ 36], there are benefits to extending the scope of student models to include additional information. Therefore, motivation, emotion, and mood could all be essential concerning student modeling.

Although even emotions researchers frequently do not agree with each other about the definitions of mood, emotion, and motivation, it would be helpful to provide a brief definition of these concepts before continuing.

According to Williams and Burden [ 37], motivation may be described as a condition of cognitive and emotional arousal, which leads to a conscious choice to take action and initiates a period of continued intellectual and/or physical effort, so as to achieve a previously determined goal. Moreover, mood and emotion have common features, but also have distinctions [ 38]. Emotion and mood share three basic characteristics: 1) They are subjective experiences, 2) they are expressed through human communicational channels, and 3) they have a physical impact. On the other hand, emotion and mood are distinct at four basic points:

  1. Duration and intensity: Duration is a characteristic of mood, while intensity is a feature of emotion.
  2. Timing: It is easier to distinguish between the beginning, climaxing, and end of an emotion than of a mood.
  3. Cause-reaction: The cause of an emotion is usually more evident than the cause of a mood. In addition, emotion triggers a target reaction, while mood frequently provokes vague reactions.
  4. Information: Emotion carries information concerning the environment, e.g., information about a threat in our environment, while mood carries information concerning our capacity to face the threat of the environment. Similarly, mood informs individuals about their progress toward personal goals [ 39], [ 40].

Mood could be useful due to its self-assessment quality. It carries descriptive information concerning a student's self-evaluation toward a learning goal. In addition, duration as a feature of mood could serve long-term learning goals. The student should have a positive attitude toward learning, both during and after interaction with the tutoring system.

Positive mood has been indicated to increase human ability to distinguish relevant from irrelevant pieces of information, thus achieving advanced performance concerning cognitive skills [ 41]. In addition, people in a positive (versus negative) mood are known to perform better on creativity tasks [ 42], [ 43]. However, positive mood is not always the optimal state for learning because it widens the thought processes, making it easier to be distracted. When the problem involves focusing, positive affect may interfere with the subject's concentration [ 44]. In such cases, negative mood could be more helpful. There is conclusive evidence that people in a negative (versus positive) mood tend to further analyze information and perform analytical/systematic information processing prior to making judgments or decisions [ 44], [ 45], [ 46]. Negative mood focuses the mind, reducing distractions. It is when the negative affect is too strong that learning tasks are inhibited [ 47].

Consequently, the further removed one is from the ideal affective state for the learning task to be accurately carried out, the more definite the impact of nonoptimal affect on performance. Thus, people experiencing positive (versus neutral) mood, for example, should be more likely to regulate mood downward when facing an analytical task, since they are further removed from the optimal negative mood [ 48].

Though Cohen and Andradea [ 48] have provided evidence that humans indeed try to self-regulate their mood to match the needs of a certain task, there are many students who cannot regulate their mood accordingly. Some children and adults have difficulty managing positive and negative affective states successfully [ 7].

An online self-assessment test could help students to regulate their mood appropriately during their preparation for exams. Thus, students would not only be cognitively but also psychologically prepared to deal with exams. Hopefully, students could use this mood regulation experience to deal with other challenging issues as well.

A first step toward this direction is to provide these systems with affect recognition techniques. Affect recognition has made remarkable progress during the last decade, but has not yet been fully adapted to intelligent tutoring systems. Improving the accuracy of recognizing people's emotions would greatly improve the likelihood of effectively integrating affect recognition methods in intelligent tutoring systems. A survey of audio-video combination efforts and a synopsis of issues in building a multimodal affect recognition system are provided by Pantic and Rothkrantz [ 49]. Preferably, evidence from many modes of interaction should be combined by a computer system so that it can generate as valid hypotheses as possible about a user's emotions. This view has been supported by several researchers in the field of human-computer interaction (HCI) [ 49], [ 50].

Humans recognize emotional states in other people by a number of visible and audible cues. Facial expression is a valuable means of communicating emotion. Moreover, there is evidence of the existence of a number of universally recognized facial expressions of emotion such as happiness, surprise, fear, sadness, anger, and disgust [ 51]. In addition, the body (gesture and posture) and tone of voice are alternative channels for communicating emotion [ 52]. There are also a number of psychophysiological correlates of emotion, such as pulse or respiration rate, most of which cannot easily be detected by human observers, but which could be made accessible to computers given appropriate sensing equipment. Through all these channels, researchers of artificial intelligence in education are attempting to infer the student's affective state. Currently, the core affect recognition methods are using personal preference information, facial expressions, physiological data, speech recognition, and questionnaire (either standalone or assisting another affect recognition method).

Questionnaires have also been used as a self-report tool for emotion. However, recently, some innovative techniques have been engaged for obtaining self-reports of emotional experience [ 53], [ 54], [ 55]. These methods indicate that stimulating the student to participate more actively in the process of self-reporting emotional experience could greatly enhance the quality of learning. For instance, Alsmeyer et al. [ 55] used the concept of color as a means of communicating emotional information. Students reported their emotions to the teacher through the selection of a color, which they had previously associated with one of seven optional emotions. In addition, they suggested that the use of color may be easier for the students to understand than the use of emotional terms themselves.

Furthermore, emotional recognition frameworks using personal preference information are based on the assumption that people do not necessarily recognize emotions just by signals seen or heard; they also use a high level of knowledge and reason, to be able to process the goals, situations, and preferences of the user. A person's emotions could be predicted if their goals and perception of relevant events were known [ 16]. Implemented in a computational model, this can be achieved by using agents, artificial intelligence techniques, reasoning on goals, situations, and preferences [ 56]. For example, if the system can reason about the reactions of a user from the input that the system receives, (assumption made derived from the time of day, reading speed, personal information provided, etc.) appropriate content could be displayed in a way adapted to the emotion or the mood of the user.

It has been demonstrated [ 32], [ 57] that it is possible to create a tutoring system able to infer a student's motivation judging from the student's interaction with the system based on a set of predefined rules. Emotion recognition systems are generally based on a rule-base system, or on a system that has learnt to solve the problem through extensive training. The richer the information provided by the interaction is, the more parameters can be derived for extracting the interaction environment and for achieving a better emotion recognition performance. This paper suggests combining various evidences in order to optimize inferences about affective states during an online self-assessment test.

With regard to learning, there have been very few approaches for the purpose of affect recognition. The adoption of affect recognition methods using personal preference information and questionnaires would probably be preferable for certain affective learning systems (e.g., Web-based for distance learning). These methods do not require any special equipment, such as video cameras, microphones, sensors, etc., thus rendering the affective learning system more user-friendly. For that reason, the method developed in this paper is based on personal preference information.

Previous Method

3.1 Starting Point

Research evidence has shown that if people are in some way inclined to regulate their mood in expectation of social interaction, the direction of such regulatory attempts should be in the direction of neutrality, regardless of whether the initial mood is positive or negative [ 58]. A neutral mood does not signify affective indifference, nor does it imply that moods are bipolar experiences. To a certain extent, it could be interpreted as readiness for participation in interaction that suspends or erases prior mood. This is because prior emotions will probably be unrelated to new interactions, so they may even disrupt them. Therefore, in anticipation of interaction, humans attempt to collect their thoughts and emotions in order to facilitate interaction.

Moreover, there is research evidence indicating that humans regard computers in a way similar to the social behavior exhibited in human-human interactions [ 59], [ 60]. Consequently, students may consider online self-assessment tests to be a form of social interaction, so they may attempt to neutralize their mood in anticipation of the test. In that sense, the model did not take into account neutral mood during the test and student's declaration of a "0 mood" was considered as a positive one. In addition, during a test, students may experience negative emotions such as anger, sorrow, despair, etc., or positive emotions such as joy, hope, pride, etc. Because of the difficulty to distinguish between each negative and positive emotion in an emotional recognition framework using personal preference information, we decided to group negative and positive emotions under negative and positive mood, respectively. Furthermore, it has been suggested that polarized moods enclose more behaviorally related information than neutral moods for the reason that the signal is stronger and probably more reliable [ 61], [ 62]. This agrees with evidence that in a positive or negative mood, compared to a neutral mood, individuals are more influenced by affect as information throughout an evaluative process [ 63].

In addition, the proposed model is largely based on the student's goal and the student's performance to achieve this goal during the test. This is consistent with research relating human goals with emotions, motivation, and mood, as discussed earlier. Moreover, each individual expects to meet a certain personalized performance goal, depending on their interest in a certain field and personal investment in time. Allowing the student to set a personal goal is tantamount to personalizing the learning experience.

3.2 Student's Mood Model

We have explored several research questions in the context of an online multiple choice questions self-assessment test, providing a measurement for evaluating students' mood during the test. One assumption was that students' goal does influence students' mood during the test in relation to the remaining questions and their record. That is to say, if a student knows that they have already failed to reach their goal during the test, because the remaining questions are fewer than the questions they have to answer correctly in order to reach their goal, then it is highly likely that they are in a negative mood. In addition to that, we assumed that student's mood is also influenced by their success or failure in answering the questions just before the current one. For instance, if a student has failed to provide a correct answer to all of the five previous questions, there is a high likelihood that they are emotionally negatively influenced, but if a student has managed to provide a correct answer to all of the five previous questions, they are highly likely to be emotionally positively influenced. In view of confirming these assumptions, we have formulated this model:

$$R(q) = N - q,\quad R(q) \in (0,N),$$


where $R$ is the number of questions remaining before the end of the test, $N$ is the total number of questions, and $q$ is the number of the current question.

$$D(q) = I - r(q),$$


where $D(q)$ is the number of questions that the student still needs to answer in order to reach their goal, $I$ is the student's goal, and $r(q)$ is the number of student's correct answers up to the current point.

$$H(q) = R(q) - D(q),$$


where $H(q)$ is a number showing whether the remaining questions are enough for the student to reach their goal. For example, $H(q) = -4$ would mean that the student has already failed to reach their goal for four questions.

$$M(q) = H(q)_{ - wr(q)}^{ + rr(q)},$$


where $M(q)$ is the student's mood, $rr(q)$ is the number of correct answers in a row just before the current question, and $wr(q)$ is the number of incorrect answers in a row just before the current question. So, if there are one or more correct answers in a row just before the current question, we add them to $H(q)$ , while if there are one or more incorrect answers in a row just before the current question, we subtract them from $H(q)$ .

New Method

4.1 Starting Point

Relationships between actual physical magnitude of a stimulus and the human perception of that stimulus were studied earlier than 1860 by the science of psychophysics [ 64]. Psychophysics has been defined as the scientific research of the relation between stimulus and sensation [ 65]. This means that it deals with the analysis of perceptual processes by observing the effect on a subject's experience or behavior of methodically varying the properties of a stimulus along one or more physical dimensions [ 66]. Psychophysics is a subdiscipline of psychology researching the connection between physical stimuli and their subjective correlates, or percepts.

Our five senses (sight, hearing, touch, taste, and smell) inform us about the surrounding environment. In other words, the most basic function of our senses is to detect energy or changes of energy in the environment (for instance, chemical as in taste or smell, electromagnetic as in vision, mechanical as in audition). According to Weber-Fechner law [ 64], as formulated in the middle of the nineteenth century, our skill to become aware of small changes in these sensations (the "just noticeable difference"), is proportional to the original intensity of the sensation.

This law was mostly founded on experiments where individuals were given two almost equal stimuli (for instance, two similar weights) and tested whether they could perceive a difference between them. It was deduced that the smallest perceptible difference was approximately proportional to the intensity of the stimulus. That is to say, if a subject could always feel that a 110 g weight was heavier than a 100 g weight, they may well also feel that 1,100 g was more than 1,000 g. What was important was that a constant relative difference in the intensity corresponds to a constant absolute difference in the logarithm of the intensity.

Similarly, other sensations work in the same way. In a quiet area, you can hear a bird singing. A second or a third bird singing could also be heard separately: The added sound would be significant in relation to the existing sound level. But the more the singing birds become, the less you are able to distinguish between their songs, because now the added sound is insignificant in relation to the already existing sound level. In other words, as sounds get louder, there needs to be a bigger alteration in intensity in order to identify it.

The mathematical description of this experience is called Stevens' Power Law [ 66]. Stevens' Power Law suggests that the perceived sensation, $R$ (loudness of a sound, brightness of a light), is an exponential function of the present level of the stimulus, $S$ , (measured sound level or brightness). It is described by the following formula:

$$R = K(S - So)^a ,$$


where $S_0$ is the threshold, or lowest noticeable level, of the sensation, and $K$ is a proportionality constant. The value of the exponent $a$ depends on the particular sensation. Using the logarithm in both sides of the equation, the formula takes the following form:

$$\log R = a\log (S - So) + \log K.$$


The evidence that human senses are better modeled through the use of an exponential function, could also apply to affective modeling. Emotions, just like senses, do not respond to stimulus in a linear way. Therefore, the new method makes the assumption that success or failure to answer correctly the just previous questions does not influence student's mood in a linear way but in a logarithmic one. The transformation of the initial model, to take into account this assumption through the integration of an exponential function, has provided a new method more efficient than the previous in estimating student's mood. To the authors' knowledge, this is the first research evidence supporting the use of logarithms to address the modeling of human affective states. Whether logarithmic expression of human emotions is a suitable way to address human affective states remains as yet to be proven.

4.2 Student's Mood Model

The previous method already indicated that taking into consideration, the recently previous correct or incorrect answers in a row just before the current question increases the method's sensitivity in evaluating student's mood. What the previous model did not take into consideration is that the effect of recently previous correct or incorrect answers in a row just before the current question may diminish as the test proceeds and these answers become less recent. According to this assumption, the recently previous correct or incorrect answers should be weighted proportionately to how recent they are.

So, instead of just adding $rr(\rm q)$ (the number of recently previous correct answers in a row just before the current question) or subtracting $wr(\rm q)$ (the number of recently previous incorrect answers in a row just before the current question) from $H(\rm q)$ , a new formula is utilized to calculate the number that is going to be added or subtracted from  $H(\rm q)$ :

$$\left[ {1 + \sum\limits_{i = 1}^{rr(q) \oplus wr(q)}{\left( {1/\exp (i)} \right)} } \right],$$


where $i$ is the number of recently previous correct or incorrect answers in a row just before the current question. Accordingly, if one or more answers just before the current question were correct, we add (7) to $H(\rm q)$ . Whereas, if one or more answers just before the current question were incorrect, we subtract (7) from $H(\rm q)$ .

The comparison between $H(\rm q)$ alone and $M(\rm q)$ ( Table 1), indicated that adding one point for each correct or incorrect answer was a useful part of the previous model. Therefore, the new model retains that part. However, it adds or subtracts only one point for all the previous correct or incorrect answers in a row before the current one. Moreover, it adds or subtracts an extra weight for each correct or incorrect answer in a row just before the current question proportionately to how recent it is.

Table 1. The Slide Bar Sequence which was Repeated Every Nine Students

The exp() function returns a number specifying $e$ (the base of natural logarithms) raised to a power. That is to say, the natural logarithm of a number is the inverse of the exp() function. The number $e$ is used to express values of such logarithmic quantities as field level, power level, sound pressure level, and logarithmic decrement [ 67]. Affective issues concerning humans could be defined as logarithmic quantities as well. We suggest that human cognitive and affective reactions are not linear to the stimulus causing these reactions, but rather exponential. Hence, in order to express the logarithmic decrement of the influence of its recently previous answer proportionately to how recent this answer is, we use the exp() function.

The model also provides minimum and maximum values for $M$ , which are obtained from the formulas shown below:

$$MaxM = N + \left[ {1 + \sum\limits_{i = 1}^N {\left( {1/\exp (i)} \right)} } \right],$$


$$MinM = - N - \left[ {1 + \sum\limits_{i = 1}^N {\left( {1/\exp (i)} \right)} } \right].$$


4.3 Calculating the Agents Feedback to Student's Mood

Since, according to (8) and (9), we know the maximum and minimum values that $M$ can take within the system, we can use a set of discrete values in order to approximate the real value of $M$ . In this way, each discrete value of input $M$ is mapped onto a discrete output value, which corresponds to a set of certain actions the agent will perform as feedback to the student. Thus, we can calculate the agent's feedback to the student using the formula shown below:

$$Feedback(M,L) = A,$$


where $L$ is the discrete level to which $M$ is assigned and $A$ is a set of actions that can arise from the $M, L$ pair. In order to provide the agent with a much richer and varied behavior, we can attach more than one possible action to each $M, L$  pair. These actions could be triggered randomly or based on the frequency of their appearance. It would be preferable if the agent would not repeat the same action for the same $M, L$ pair. Thus, the system would not repeat itself each time the students were in a certain mood, making the students bored.

Emotional feedback can be implemented by using beneficially positive emotions, while preventing, controlling, and managing negative emotions. Moreover, the emotional feedback can also be implemented using negative emotions in order to increase the student's devotion and engagement. These "strategies" can be applied using humor and jokes, amusing games, expressions of sympathy, reward, pleasant surprises, encouragement, acceptance, praises but also through criticism and punishment [ 11]. Nevertheless, further research is needed in order to define an effective emotional feedback plan.


High school students ( $N = 153$ ) were recruited from three different regions of Greece (60 students from Athens, 50 students from Thessaloniki, and 43 students from Mitilini). Respondents consisted of 56 percent females and 44 percent males. The average age of participants was 16.8 ( ${\rm SD} = 1.98$ ), with 90 percent of the sample ranging from 15 to 19 years. So as to serve the needs of the experiment, an online multiple choice questions test was built within a Windows XP machine using JavaScript with Perl CGI on Apache Web server with MySQL.

5.1 Items

The multiple choice questions were focused on basic computer knowledge and skills, based on material taught in lectures. The context of questions was prespecified by the teachers prior to the study. The test was composed of 45 questions. The order of questions presented was randomly altered among students.

5.2 Procedure and Data Collection Methodology

The duration of the experiment was approximately 45 minutes and took place during the regular schedule of laboratorial classes. Students were told that that was a general education test concerning computer knowledge that would help them assess their knowledge about computers. At the beginning of the test, the system asked students how many correct answers would make them feel satisfied with the level of their knowledge, having them set their personal goal. Throughout the test, each student selected their answer among four possible answers and confirmed their choice by clicking the "submit" button. After each question, the system informed the student whether their answer was right or wrong and presented their score. Then, the student could proceed to the next question by clicking the "next" button.

During these 45 questions, a slide bar ( Fig. 1) appeared, asking the student to move it according to their mood concerning the test, from $-100$ (extremely negative mood) to +100 (extremely positive mood). In order not to irritate or distract students, the slide bar appeared five times during the test, once every nine questions, at a different instant for each student. Accordingly, it took nine students for the slide bar to appear once after every question of the test ( Table 1). So, the 153 participants gave us the chance to check students' mood after every question 17 times. Thus, the data set consisted of 765 instances (five instances of each student).

Graphic: Fig. 1. A screenshot of the slide bar.

Figure    Fig. 1. A screenshot of the slide bar.

Each time the student declared their mood by moving the slide bar, 10 parameters were calculated and recorded:

  1. the number of the current question,
  2. the number of questions remaining before the end of the test,
  3. the number of questions that the student still needs to answer in order to reach his goal,
  4. the number that shows whether the remaining questions are more or fewer than the number of questions that the student has to answer so as to reach his goal, i.e., student's hope to accomplish their goal,
  5. the number of correct answers in a row before the current question,
  6. the number of incorrect answers in a row before the current question,
  7. the number of correct answers up to the current question,
  8. the number of incorrect answers up to the current question,
  9. the score, and
  10. the mood that the student indicated by moving the slide bar.


To show the distribution of the indicated mood on the slide bar, a percentile plot is used ( Fig. 2), since percentiles are very helpful in exploring distribution of numerical sets. So, the indicated mood sample is nearly Gaussian, but with some outliers. The mean is 12.925 and the standard deviation 37.1. There is some likelihood, called the confidence level, that the true population mean error falls within a particular range, called the confidence interval, around the mean error value obtained from our sample. A confidence level of 90 percent gives a confidence interval of 4.89, which means that the range for the true population mean is between 8.04 and 17.82. This is observed mood variability for a sample that is nearly Gaussian.

6.1 Results—Previous Method

Initially, we evaluated the model based on its divergence from the mood that students pointed out on the slide bar. The mean error was normalized from 0 to 1. Nevertheless, this is a quantitative way of evaluation, while it is difficult to be highly accurate in predicting students' mood, which is an extremely sensitive variable.

Although this is a quantitative evaluation, the linear correlation coefficient between user-declared mood and mood estimated by the model, is strong ( $r = 0.71$ ). Experiments in HCI involve people, who would be unpredictable. Thus, we consider that a mean error of 0.1, normalized from 0 to 1, with a standard deviation of 0.1 and having a normal error distribution is a fairly good performance in predicting a student's mood. A confidence level of 90 percent gives us a confidence interval of 0.01, which means that the range for true population mean error is 0.09-0.11.

It is obvious that the closer to their goal the student is throughout the test, the more positive their mood becomes. At the other extreme, the more the student's distance from their goal widens as they proceed into the test, the more negative their mood becomes.

Another important issue is that adding or subtracting the number of correct or incorrect answers in a row just before the current question increases the model's efficacy ( Table 2). We compared $H(q)$ alone with the entire model, which is $H(q)$ plus $rr(q)$ (the number of correct answers in a row just before the current question) or minus $wr(q)$ (the number of incorrect answers in a row just before the current question). This shows that student's success or failure to recently previous questions can influence their mood positively or negatively toward the current question.

Table 2. Comparing $H(q)$ with $M(q)$ Shows that Taking into Account the Number of Correct or Incorrect Answers in a Row Just Before the Current Question Increases the Model's Efficacy

6.2 Results—New Method

The linear correlation coefficient between user-declared mood and mood estimated by the new method is stronger than that of the previous method ( $r = 0.73$ ). The mean error for the new method is also improved at 0.05, with a standard deviation of 0.13, and has a normal error distribution. A confidence level of 90 percent for the new model gives us a confidence interval of 0.02, which means that the range for true population mean is 0.03-0.07.

When we try to determine whether a student is in a positive mood or in a negative mood, the new method is again more successful than the previous one ( Table 3).

Table 3. Comparing the Previous Method with the New Method Indicates that Weighting the Recently Previous Correct or Incorrect Answers, Proportionally to How Recent They Are, Increases the Model's Efficacy

Modeling multiple variables is important as students have complex characteristics that ultimately affect their performance. However, adding additional variables will not always increase the accuracy of the student model [ 68]. In this case, the choice of taking into account how recent a student's correct or incorrect answer is showed good results. It seems that the influence of correct or incorrect answers fades as the test proceeds and what counts more is the most recent record.

The variables that have an influence on student's mood while taking an online self-assessment test should be verified by further research. Furthermore, it is not just the variables that help to predict student's mood, but their combination in adequate formulae is what makes the difference.

Table 4. An Example Sequence of Questions and Answers

Advantage of the New Method over the Previous Method

Let us assume that a student named John has set a personal goal to answer correctly 31 out of 45 questions. Moreover, he has already provided answers to 35 questions, 20 correct, and 15 incorrect. Additionally, he has provided a correct answer to the four previous questions in a row (questions 32-35). Thus, he is now ready to deal with question number 36. Using the previous method, John's mood at that point would be:

$$R(35) = 45 - 35 \Rightarrow R(35) = 10,$$


$$D(35) = 31 - 20 \Rightarrow D(35) = 11,$$


$$H(35) = 10 - 11 \Rightarrow H(35) = - 1,$$


$$M(35) = - 1 + 4 \Rightarrow M(35) = 3.$$


The above could be considered as a positive mood, although John is already one question behind achieving his personal goal. According to the new method, John's mood would be:

$$\eqalign{M(35) &= - 1 + [1 + (1/\exp (1)) + (1/\exp (2))\cr\noalign{\vskip4pt} &\quad + (1/\exp (3)) + (1/\exp (4))] \cr\noalign{\vskip4pt} &\Rightarrow M(35) = - 1 + 1.57 \Rightarrow M(q) = 0.57. }$$


Now, let us examine what the two methods would have predicted for John's mood given the following sequence of questions and answers ( Table 4):

Previous method question 1:

$$R(1) = 45 - 1 \Rightarrow R(q) = 44,$$


$$D(1) = 31 - 0 \Rightarrow D(q) = 31,$$


$$H(1) = 44 - 31 \Rightarrow H(q) = 13,$$


$$M(1) = 13 - 1 = 12.$$


New method question 1:

$$M(1) = 13 - [1 + (1/\exp (1))] \Rightarrow M(1) = 11.63.$$


Previous method question 2:

$$R(2) = 45 - 2 \Rightarrow R(2) = 43,$$


$$D(2) = 31 - 1 \Rightarrow D(2) = 30,$$


$$H(q) = 43 - 30 \Rightarrow H(q) = 13,$$


$$M(2) = 13 + 1 \Rightarrow M(q) = 14.$$


New method question 2:

$$M(2) = 13 + [1 + (1/\exp (1)] \Rightarrow M(2) = 14.38.$$


Previous method question 3:

$$R(3) = 45 - 3 \Rightarrow R(3) = 42,$$


$$D(3) = 31 - 2 \Rightarrow D(3) = 29,$$


$$H(3) = 42 - 29 \Rightarrow 13,$$


$$M(3) = 13 + 2 \Rightarrow M(q) = 15.$$


New method question 3:

$$\eqalign{M(3) &= 13 + [1 + (1/\exp (1)) + (1/\exp (2))] \cr &\Rightarrow M(3) = 14.5. }$$


Previous method question 4:

$$R(4) = 45 - 4 \Rightarrow R(4) = 41,$$


$$D(4) = 31 - 3 \Rightarrow D(4) = 28,$$


$$H(4) = 41 - 28 \Rightarrow H(4) = 13,$$


$$M(4) = 13 + 3 \Rightarrow M(q) = 16.$$


New method question 4:

$$\eqalign{M(4) &= 13 + [1 + (1/\exp (1)) + (1/\exp (2)) + (1/\exp (3))] \cr &\Rightarrow M(4) = {\rm 14}{\rm .55}{\rm .} }$$


Previous method question 5:

$$R(5) = 45 - 5 \Rightarrow R(5) = 40,$$


$$D(5) = 31 - 3 \Rightarrow D(5) = 28,$$


$$H(5) = 40 - 28 \Rightarrow H(5) = 12,$$


$$M(5) = 12 - 1 \Rightarrow M(5) = 11.$$


New method question 5:

$$M(5) = 12 - [1 + (1/\exp (1))] \Rightarrow M(5) = 10.63.$$


Previous method question 6:

$$R(6) = 45 - 6 \Rightarrow R(6) = 39,$$


$$D(6) = 31 - 3 \Rightarrow D(6) = 28,$$


$$H(6) = 39 - 28 \Rightarrow H(6) = 11,$$


$$M(6) = 11 - 2 \Rightarrow M(6) = 9.$$


New method question 6:

$$\eqalign{M(6) &= 11 - [1 + (1/\exp (1)) + (1/\exp (2))] \cr &\Rightarrow M(6) = 9.5. }$$


Previous method question 7:

$$R(7) = 45 - 7 \Rightarrow R(7) = 38,$$


$$D(7) = 31 - 3 \Rightarrow D(7) = 28,$$


$$H(7) = 38 - 28 \Rightarrow H(7) = 10,$$


$$M(7) = 10 - 3 \Rightarrow M(7) = 7.$$


New method question 7:

$$\eqalign{M(7) &= 10 - [1 + (1/\exp (1)) + (1/\exp (2)) + (1/\exp (3))] \cr &\Rightarrow M(7) = 8.44. }$$


Previous method question 8:

$$R(8) = 45 - 8 \Rightarrow R(8) = 37,$$


$$D(8) = 31 - 3 \Rightarrow D(8) = 28,$$


$$H(8) = 37 - 28 \Rightarrow H(8) = 9,$$


$$M(8) = 9 - 4 \Rightarrow M(8) = 5.$$


New method question 8:

$$\eqalign{M(8) &= 9 - [1 + (1/\exp (1)) + (1/\exp (2))\cr &\quad + (1/\exp (3)) + (1/\exp (4))] \cr &\Rightarrow M(8) = 7.42. }$$


Using John's case to display the example of Table 4, reveals that the new method behaves more smoothly than the previous ( Fig. 3). In order to demonstrate the advantage of the new method over the previous method, it is crucial to examine how the two methods operate under "extreme conditions." Let us assume that a test is constituted of 100 questions and that a student named Mark has set his personal goal to provide correct answers to all of the 100 questions. Let us also assume that Mark has totally failed to reach this goal. He has answered incorrectly 99 questions in a row. According to the previous method, Mark's mood at that point would be:

$$R(99) = 100 - 99 \Rightarrow R(99) = 1,$$


$$D(99) = 100 - 0 \Rightarrow D(99) = 100,$$


$$H(99) = 1 - 100 \Rightarrow H(99) = - 99,$$


$$M(99) = - 99 - 99 \Rightarrow M(99) = - 198.$$


Supposing that Mark answers the last question of the test correctly, the previous method would predict Mark's mood as follows:

$$R(100) = 100 - 100 \Rightarrow R(100) = 0,$$


$$D(100) = 100 - 1 \Rightarrow D(100) = 99,$$


$$H(100) = 0 - 99 \Rightarrow H(100) = - 99,$$


$$M(100) = - 99 + 1 \Rightarrow M(100) = - 98.$$


Based on the previous method, Mark's mood would be ameliorated by 100 units, from $-198$ to $-98$ , just because he answered the last question of the test correctly. This is a very unrealistic prediction. Predicting Mark's mood with the new method leads to the solution of this problem as follows:

For question 99:

$$M(99) = - 100.58.$$


And for question 100:

$$M(100) = - 97.63.$$


Therefore, the new method provides a much more realistic prediction under "extreme conditions." The new method agrees with the previous on the fact that the student's success or failure to the previous questions does have an influence on student's mood. Nevertheless, the new method weights this influence differently. The less recent a question is, the less its influence on student's current mood should be weighted. This feature allows the new method to adapt to unexpected testing conditions, such as Mark's "extreme case."


Figure    Fig. 2. Percentile plot of the mood indicated on the slide bar.


Figure    Fig. 3 An example of the previous and the new method.

The new method manages to adapt not only to the ordinary student but also to the unanticipated one. However, the difference between the two methods is not so apparent and did not have a strong effect for the study reported, since it does not concern the usual student type. Nevertheless, in unexpected cases, the previous method fails by far to predict student's mood. Thus, the difference between the two methods may be small for a whole group of students, but it is very essential for those individuals at the extremes. Consequently, the new method is preferable as it incorporates all the advantages of the previous method plus an apparent advantage at extreme cases. Personalizing the learning experience through a tutoring system requires building models able to adapt to a wide variety of characters beyond the usual student type. The new method can support the needs of personalized self-assessment much more efficiently than the previous one.


This paper has presented formulas for recognition of student mood during an online self-assessment test. Additionally, it has argued that the assumptions underlying the formulas may prove useful for future research.

The presented work aims to provide tutoring systems with mood recognition methods for use during an online self-assessment test. The proposed methods are easy to implement in a system. So far, there has been no applicable computational model for affect recognition during an online test.

The two methods were examined for their reliability based on the cognitive area of information technology. Most likely, the two methods can be applied to other cognitive areas and be as effective as they were in this experiment. In any case, new experiments are needed in order to confirm this assumption.

One essential part of both methods, which proved to be crucial, is that students set their personal performance goal at the beginning of the test. This feature could help self-assessment tests to adapt to the specific personal characteristics of each student, such as their interests, ambitions, personal experiences, time devoted to studying, etc. Moreover, setting a personal performance goal may enhance self-motivation and self-awareness. Therefore, it may be important to start building future systems taking into account student's personal goal.

In addition, the present study has indicated that it may be useful to integrate "exponential logic" into affective modeling. Since human senses are better modeled with the use of logarithms, affective issues could be modeled more effectively with the use of logarithms as well. Furthermore, the new method introduced in this paper has suggested that the influence of questions already answered should be weighted in relation to their distance from the current question. Thus, indicating that the influence of previous questions on student's mood diminishes as new questions appear. As a result, using the new method to infer a student's mood, a higher level of personalization can be achieved.

The methods described in this paper could be used as a diagnostic means for student mood during online self-assessment tests that would help the student both psychologically and cognitively. Obviously, these methods do not structure an efficient system in their own right. Other emotional and motivational methods should be used as well to inform the student's model. In addition, research concerning emotional feedback would help to determine the proper system response to a student's recognized emotional and motivational states based on integrated pedagogical models.

The experience that is going to be derived from the implementation of such methods in tutoring systems will reveal their strengths and weaknesses. For the time being, integrating affect recognition methods into tutoring systems makes personalized feedback much more effective. This yields tutoring systems that are more efficient than just applying a nonindividualized feedback as was done in previous e-learning tools.


The authors wish to thank the Alexander S. Onassis Public Benefit Foundation for the grant of a doctorate scholarship to the first author.


About the Authors

Bio Graphic
Christos N. Moridis received the BS degree in communication, media, and culture from the Panteion University of Athens in 2004 and the MSc degree in advanced systems of computing and communications specializing in intelligent systems from the Department of Electrical and Computing Engineering, Aristotle University of Thessaloniki, in 2007. Currently, he is working toward the PhD degree in the Information Systems Department at the University of Macedonia, Thessaloniki, Greece. His research interests include affective learning systems, emotional agents, fuzzy systems, interface design, and search engines.
Bio Graphic
Anastasios A. Economides received the Dipl Eng degree in electrical engineering from the Aristotle University of Thessaloniki in 1984. Holding a Fulbright and a Greek State fellowship, he received the MSc and PhD degrees in computer engineering from the University of Southern California, Los Angeles, in 1987 and 1990, respectively. At graduation, he received the outstanding academic achievement award from the University of Southern California. Currently, he is an associate professor and chairman of the Information Systems Department at the University of Macedonia, Thessaloniki, Greece. He is the director of Computer Networks and Telematics Applications (CONTA) Laboratory. His research interests include e-learning, e-services, and networking techno-economics. He has published more than 150 peer-reviewed papers. He has been the plenary speaker in two international conferences. He has served on the editorial board of several international journals, on the program committee of many international conferences, and as a reviewer for many international journals and conferences. Finally, he has been the principal investigator of several funded projects and participated in many funded projects.
73 ms
(Ver 3.x)