Studies in cognitive science have found that interactive robots, interactive games, etc., may help the emotional development of children with mental-health problems (e.g. autistic children). It is very important with these children to show empathic behaviors and to take into account their emotion when relating to them. Current technology cannot do this because it cannot recognize the emotion of the children. Few studies are directed toward this goal by categorizing affective behavior of the child into a set of discrete categories. But still two problems exist: gesture is not yet a concern as channel of affective communication in interactive technology, and existing systems only model discrete categories, but not affective dimensions, e.g., intensity. In this work, we propose a multi-agent based interactive system that can quantify child?s performance (emotion with intensity of emotion) in real time.