This Article 
 Bibliographic References 
 Add to: 
Automatic Support for Usability Evaluation
October 1998 (vol. 24 no. 10)
pp. 863-888

Abstract—The main goal of this work is to propose a method to evaluate user interfaces using task models and logs generated from a user test of an application. The method can be incorporated into an automatic tool which gives the designer information useful to evaluate and improve the user interface. These results include an analysis of the tasks which have been accomplished, those which failed and those never tried, user errors and their type, time-related information, task patterns among the accomplished tasks, and the available tasks from the current state of the user session. This information is also useful to an evaluator checking whether the specified usability goals have been accomplished.

[1] N. Bevan, "Measuring Usability as Quality of Use," Software Quality J. vol. 4, pp. 115-130. Chapman&Hall, 1995.
[2] S.K. Card, T.P. Moran, and A. Newell, The Psychology of Human-Computer Interaction, Erlbaum, Hillsdale, N.J., 1983.
[3] D. Diaper, Task Analysis for Human-Computer Interaction.Chichester: Ellis Horwood, 1989.
[4] J.S. Dumas and J.C. Redish, A Practical Guide to Usability Testing.Norwood, N.J.: Ablex, 1994.
[5] M. Good et al., "User-Derived Impact Analysis as a Tool for Usability Engineering," Proc. CHI Conf. Human Factors in Computing Systems, ACM Press, New York, 1986, pp. 241-246.
[6] D. Hix and H. Hartson, Developing User Interfaces: Ensuring Usability through Product&Process, John Wiley&Sons, New York, 1993.
[7] ISO Information Processing Systems—Open Systems Interconnection—LOTOS—A Formal Description Based on Temporal Ordering of Observational Behavior, ISO/IS 8807. ISO Central Secretariat, 1988.
[8] R. Jeffries, J.R. Miller, C. Wharton, and K.M. Uyeda, "User Interface Evaluation in the Real World: A Comparison of Four Techniques," Proc. Conf. Human Factors in Computing Systems, CHI'91, pp. 119-124, ACM Press, 1991.
[9] D.E. Kieras, S.D. Wood, K. Abotel, and A. Hornof, "GLEAN: A Computer-Based Tool for Rapid GOMS Model, Usability Evaluation of User Interface Designs," Proc. UIST'95, pp. 91-100,New York: ACM Press, 1995.
[10] D. Kieras, W. Scott, and D. Meyer, "Predictive Engineering Models Based on the EPIC Architecture for a Multimodal High-Performance Human-Computer Interaction Task," ACM Trans. Computer-Human Interaction, vol. 4, no. 3, pp. 230-275, Sept. 1997.
[11] C. Lewis, "Using the 'Thinking Aloud' Method in Cognitive Interface Design," Research Report RC9265, IBM Thomas J. Watson Research Center, Yorktown Heights, N.Y., 1982.
[12] C. Lewis and J. Rieman, Task-Centered User Interface Design: A Practical Introduction, a shareware book published on the web by the TextVersionindex.htm. Also available HCI-Design-Book, 1993.
[13] M. Macleod, R. Bowden, and N. Bevan, "The MUSiC Performance Measurement Method," HCI'96, Tutorial 14, Measuring Usability—MUSiC Methods,N. Bevan, London: The British HCI Group, 1994.
[14] J. Nielsen, Usability Engineering, Academic Press, New York, 1993.
[15] S. Pangoli and F. Paternò, "Automatic Generation of Task-Oriented Help," Proc. ACM Symp. User Interfaces Software and Technology., pp. 181-187,Pittsburgh: ACM Press, 1995.
[16] F. Paternò, C. Mancini, and S. Meniconi, "ConcurTaskTrees: A Diagrammatic Notation for Specifying Task Models," Proc. Interact'97, pp. 362-369,Sydney: Chapman&Hall, 1997.
[17] F. Paternò, C. Mancini, and S. Meniconi, "Engineering Task Models," IEEE Conf. Eng. Complex Systems, pp. 69-76,Como, IEEE CS Press, Sept. 1997.
[18] F. Paternò, M.S. Sciacchitano, and J. Löwgren, "A User Interface Evaluation Mapping Physical User Actions to Task-Driven Formal Specifications," Design, Specification and Verification of Interactive Systems'95. Proc. Eurographics Workshop, P. Palanque and R. Bastide, eds., pp. 35-53,Toulouse: Springer-Verlag, 1995.
[19] P.G. Polson, C. Lewis, J. Rieman, and C. Wharton, "Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces," Int'l J. Man-Machine Studies, vol. 36, pp. 741-773, 1992.
[20] Human-Computer Interaction, J. Preece, Y. Rogers, H. Sharp, D. Benyon, S. Holland, and T. Carey, eds. Workingham, England: Addison-Wesley, 1994.
[21] A. Sears, "Layout Appropriateness: A Metric for Evaluating User Interface Widget Layout," IEEE Trans. Software Eng., vol. 17, no. 7, pp. 707-719, July 1993.
[22] A. Sears, "AIDE: A Step Toward Metric-Based Interface Development Tools," Proc. 8th ACM Symp. User Interface Software and Technology, ACM Press, New York, 1995, pp. 101-110.
[23] C. Wharton, J. Rieman, C. Lewis, and P. Polson, "The Cognitive Walkthrough: A Practitioner's Guide," Usability Inspection Methods, J. Nielsen and R.L. Mack, eds. New York: John Wiley&Sons, 1994.
[24] S. Wilson, P. Johnson, C. Kelly, J. Cunningham, and P. Markopoulos, "Beyond Hacking: A Model-Based Approach to User Interface Design," Proc. HCI'93. J.L. Alty et al., eds., People and Computers VIII, Proc. Conf., HCI'93 Loughborough, UK, Cambridge: CUP, 1993.

Index Terms:
User interfaces, usability engineering, formal methods for human-computer interaction, task models, user interface evaluation.
Andreas Lecerof, Fabio Paternò, "Automatic Support for Usability Evaluation," IEEE Transactions on Software Engineering, vol. 24, no. 10, pp. 863-888, Oct. 1998, doi:10.1109/32.729686
Usage of this product signifies your acceptance of the Terms of Use.