The Community for Technology Leaders
RSS Icon
Issue No.12 - Dec. (2012 vol.18)
pp: 2869-2878
Youn-ah Kang , Google Inc.
John Stasko , Georgia Institute of Technology
While the formal evaluation of systems in visual analytics is still relatively uncommon, particularly rare are case studies of prolonged system use by domain analysts working with their own data. Conducting case studies can be challenging, but it can be a particularly effective way to examine whether visual analytics systems are truly helping expert users to accomplish their goals. We studied the use of a visual analytics system for sensemaking tasks on documents by six analysts from a variety of domains. We describe their application of the system along with the benefits, issues, and problems that we uncovered. Findings from the studies identify features that visual analytics systems should emphasize as well as missing capabilities that should be addressed. These findings inform design implications for future systems.
Visual analytics, Qualitative analysis, Electronic mail, Data visualization, Market research, qualitative evaluation, Visual analytics, case study
Youn-ah Kang, John Stasko, "Examining the Use of a Visual Analytics System for Sensemaking Tasks: Case Studies with Domain Experts", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 12, pp. 2869-2878, Dec. 2012, doi:10.1109/TVCG.2012.224
[1] C. Andrews and C. North., Analyst's Workspace: An Embodied Sensemaking Environment for Large, High Resolution Displays. To appear in IEEE VAST ‘12. etd-082320 11–1603501
[2] E. Bier, E. Ishak, and E. Chi, Entity Workspace: an evidence file that aids memory, inference, and reading Proceedings of the IEEE International Conference on Intelligence and Security Analysis, pp. 466-472, May 2006.
[3] E. Bier, S. Card, and J. Bodnar,Entity-based collaboration tools for intelligence analysis. Proceedings of IEEE VAST, pp. 99-106, Oct. 2008.
[4] Balie. http:/ 01 March 2012).
[5] E. Braunstein,C. Görg, Z. Liu, and J. Stasko., Jigsaw to Save Vastopolis - VAST 2011 Mini Challenge 3 Award: “Good Use of the Analytic Process”, Proceedings of IEEE VAST ‘11, Providence, RI, pp. 323-324, October 2011.
[6] A.R. Chappell, A.J. Cowell, D.A. Thurman,, and J.R. Thomson., Supporting mutual understanding in a visual dialogue between analyst and computer. Proceedings of Human Factors and Ergonomic Society 48th Annual Meeting (New Orleans, LA), Human Factors and Ergonomics Society: Santa Monica, CA, pp. 376-380, September 2004.
[7] K. Charmaz., Qualitative interviewing and grounded theory analysis. In Handbook of Interview Research: Context and Method,( J. F. Gubrium, and J. A. Holstein, eds.), CA: Sage Publications, 2002.
[8] C. Chen, and M. Czerwinski, (Eds.) Introduction to the Special Issue on Empirical evaluation of information visualizations. International Journal of Human-Computer Studies, 53, 5, pp. 631-635, 2000.
[9] G. Chin, O.A. Kuchar, and K.E. Wolf, Exploring the analytical processes of intelligence analysts Proceedings of ACM CHI, pp. 11-20, April 2009.
[10] Cicero. Language Computer Corp. http:/ (accessed 01 March 2012).
[11] Compendium. http:/ (accessed 11 July 2012).
[12] GATE — General Architecture for Text Engineering. http:/ (accessed 01 March 2012).
[13] C. Görg, Z. Liu, N. Parekh., K. Singhal, and J. Stasko., Jigsaw meets Blue Iguanodon - The VAST 2007 Contest, (Contest Paper), Proceedings of IEEE VAST ‘07, Sacramento, CA, pp. 235-236, October 2007.
[14] D. Gotz,M. X. Zhou,, and Z. Wen., A study of information gathering and result processing in intelligence analysis. In IUI 2006 Workshop on IUI for Intelligence Analysis, 2006.
[15] E. Hetzler and A. Turner, Analysis experiences using information visualization IEEE Computer Graphics and Applications, pp. 22-26, 2004.
[16] J. Hughes, V. King, T. Rodden,, and H. Andersen., The role of ethnography in interactive systems design, ACM interactions, 2, 2, pp. 56-65, April 1995.
[17] J. Hughes,J. O’Brien, T. Rodden, and M. Rouncefield., Design with ethnography: A presentation framework for design, Proc. of Design of Interaction Systems ‘97, ACM Press, New York, pp. 147-159, 1997.
[18] i2 — Analyst's Notebook. http://www.i2group.comus(accessed 11 July 2012).
[19] Jigsaw tutorial. (accessed 11 March 2012).
[20] Y. Kang,C. Görg,, and J. Stasko., The evaluation of visual analytics systems for investigative analysis: Deriving design principles from a case study. Proceedings of IEEE VAST, pp. 139-146, October 2009.
[21] Y. Kang,C. Görg,, and J. Stasko., How Can Visual Analytics Assist Investigative Analysis? Design Implications from an Evaluation. IEEE Transactions on Visualization and Computer Graphics, 17, 5, pp. 570-583, May 2011.
[22] Y. Kang and J. Stasko, Characterizing the Intelligence Analysis Process: Informing Visual Analytics Design through a Longitudinal Field Study Proceedings of IEEE VAST ’11, Providence, RI, pp. 21-30, October 2011.
[23] Y. Kang., Informing Design of Visual Analytics Systems for Intelligence Analysis: Understanding User, User Tasks, and Tool Usage, PhD Dissertation, School of Interactive Computing, Georgia Institute of Technology, 2012.
[24] A. Komlodi, A. Sears, and E. Stanziola., Information Visualization Evaluation Review. ISRC Tech. Report, Dept. of Information Systems, UMBC.UMBC- ISRC-2004– 1.
[25] H. Lam, E. Bertini, P. Isenberg., C. Plaisant, and S. Carpendale, Empirical Studies in Information Visualization: Seven Scenarios IEEE Transactions on Visualization and Computer Graphics, 2012. Appeared online: 30 November 2011.
[26] A. Perer and B. Shneiderman,Integrating statistics and visualization: case studies of gaining clarity during exploratory data analysis. Proceedings of ACM CHI, pp. 265-274, April 2008.
[27] O. J. Pinon,D. N. Mavris,, and E. Garcia., A Visual Analytics Approach to the Qualitative Comparison of the SESAR and NextGen Efforts 9th AIAA Aviation Technology Integration and Operations ATIO Conference, 2009.
[28] C. Plaisant, The challenge of information visualization evaluation Proceedings of AVI, pp. 109-116, May 2004.
[29] C. Plaisant, G. Grinstein, J. Scholtz., M. Whiting, T. O’Connell, S. Laskowski, L. Chien., A. Tat, W. Wright,C. Görg, Z. Liu, N. Parekh., K. Singhal, and J. Stasko., Evaluating Visual Analytics at the 2007 VAST Symposium Contest, IEEE Computer Graphics and Applications, 28.2, pp. 12-21. March 2008.
[30] Research and Development Descriptive Summaries (RDDS). rdds _ desc.html (accessed February 8, 2012).
[31] A. Rose, C. Plaisant, and B. Shneiderman., Using ethnographic methods in user interface re-engineering, Proceedings of DIS ‘95: Symposium on Designing Interactive Systems, ACM Press, New York, pp. 115-122, 1995.
[32] A. Sanfilippo, B. Baddeley, A.J. Cowell, M.L. Gregory, R. Hohimer,, and S. Tratz., Building a human information discourse interface to uncover scenario content. Proceedings of the 2005 International Conference on Intelligence Analysis (McLean, VA), May 2005.
[33] P. Saraiya, C. North, and K. Duca, An evaluation of micro array visualization tools for biological insight Proceedings of the Symposium on Information Visualization ‘04(Los Alamitos, USA), pp. 1-8, 2004.
[34] P. Saraiya, C. North, V. Lam,, and K.A. Duca., An Insight-Based Longitudinal Study of Visual Analytics, IEEE Transactions on Visualization and Computer Graphics, 12, 6, pp. 1511-1522, November 2006.
[35] J. Scholtz, Beyond usability: Evaluation aspects of visual analytic environments IEEE VAST, pp. 145-150, October 2006.
[36] J. Scholtz, Developing guidelines for assessing visual analytics environments Information Visualization, 10, 3, pp. 212-231, 2011.
[37] A. Selvin and S. B. Shum., Narrative, sensemaking, and improvisation in participatory hypermedia construction CHI ‘08 Workshop on Sensemaking, Florence, Italy, 2008.
[38] J. Seo and B. Shneiderman, Knowledge discovery in high dimensional data: Case studies and a user survey for the rank-by feature framework IEEE Transactions on Visualization and Computer Graphics, 12(3), pp. 311-322, 2006.
[39] B. Shneiderman and C. Plaisant, Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies Proceedings of ACM BELIV, pp. 1-7, May 2006.
[40] J. Stasko, C. Görg, and Z. Liu, Jigsaw: supporting investigative analysis through interactive visualization Information Visualization, 7, 2, pp. 118-132, 2008.
[41] A. Strauss and J. Corbin., Basics of qualitative research: Grounded theory procedures and techniques. Sage Publications: Newbury Park, Calif, 1990.
[42] W. Wright, D. Schroh, P. Proulx., A. Skaburskis, and B. Cort, The Sandbox for analysis: concepts and methods Proceedings of ACM CHI ‘06 (Montreal, Quebec), pp. 801-810, April 2006.
23 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool