Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 1 (AAMAS'04)
An Empirical Study of the Effect of Agent Competence on User Performance and Perception
New York City, New York, USA
July 19-July 23
ISBN: 0-7695-2092-8
We studied the role of the competence of an interface agent that helped users to learn and use a text editor. Participants in the study made a set of changes to a document with the aid of one of four interface agents. The agent would respond to participants? spoken questions as well as make proactive suggestions using a synthesized voice. The agents varied in the quality of responses and suggestions. One group of participants could also access a help screen. Results revealed that the perceived utility of the agent was influenced by the types of errors it made, while participants' subjective impressions of the agent related to the perceptions of its embodiment. Additionally, allowing participants to choose their preferred assistance styles improved objective performance. We relate quantitative findings with qualitative interview data and discuss implications for the design and the implementation of systems with interface agents.
Index Terms:
Information interface and presentation, User Interfaces, Evaluation/methodology, Embodied conversational agents, interface agents, empirical evaluation, Wizard-of-Oz technique, qualitative study
Citation:
Jun Xiao, John Stasko, Richard Catrambone, "An Empirical Study of the Effect of Agent Competence on User Performance and Perception," aamas, vol. 1, pp.178-185, Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 1 (AAMAS'04), 2004