| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
An Insight-Based Longitudinal Study of Visual Analytics
November/December 2006 (vol. 12 no. 6)
pp. 1511-1522
Abstract—Visualization tools are typically evaluated in controlled studies that observe the short-term usage of these tools by participants on preselected data sets and benchmark tasks. Though such studies provide useful suggestions, they miss the long-term usage of the tools. A longitudinal study of a bioinformatics data set analysis is reported here. The main focus of this work is to capture the entire analysis process that an analyst goes through from a raw data set to the insights sought from the data. The study provides interesting observations about the use of visual representations and interaction mechanisms provided by the tools, and also about the process of insight generation in general. This deepens our understanding of visual analytics, guides visualization developers in creating more effective visualization tools in terms of user requirements, and guides evaluators in designing future studies that are more representative of insights sought by users from their data sets.
[1] 1511 C. Plaisant, “The Challenge of Information Visualization Evaluation,” Proc. Conf. Advanced Visual Interfaces (AVI '04), 2004.[2] C. Chen and M. Czerwinski, “Empirical Evaluation of Information Visualizations: An Introduction,” Int'l J. Human-Computer Studies, vol. 53, pp 631-635, 2000.[3] A. Kobsa, “An Empirical Comparison of Three Commercial Information Visualization Systems,” Proc. IEEE InfoVis Conf., pp.123-130, 2001.[4] P. Saraiya, C. North, and K. Duca, “An Insight-Based Methodology for Evaluating Bioinformatics Visualization,” IEEE Trans. Visualization and Computer Graphics, vol. 11, no. 4, July/Aug. 2005.[5] D. Duggan, B. Bittner, Y. Chen, P. Meltzer, and J. Trent, “Expression Profiling Using cDNA Microarrays,” Nature Genetics, vol. 21, pp. 11-19, Jan. 1999.[6] L. Shi, “DNA Microarray—Genome Chip,” http://www.gene-chips.comGeneChips.html#What , 2002.[7] R. Spence, Information Visualization. Addison-Wesley, 2001.[8] S. Card, J.D. Mackinlay, and B. Shneiderman, Readings in Information Visualization—Using Visualization to Think. San Francisco: Morgan Kaufmann, 1999.[9] C. Chen and Y. Yu, “Empirical Studies of Information Visualization: A Meta-Analysis,” Int'l J. Human-Computer Studies, vol. 53, pp. 851-866, 2000.[10] P. Irani and C. Ware, “Diagramming Information Structures Using 3D Perceptual Primitives,” ACM Trans. Computer-Human Interaction, vol. 10, no. 1, pp. 1-19, 2003.[11] H. Hartson and D. Hix, Developing User Interfaces: Ensuring Usability through Product and Process. John Wiley, 1993.[12] G. Rao and D. Mingay, “Report on Usability Testing of Census Bureau's Dynamaps CD-ROM Product,” http://infovis.cs.vt.edu/cs5764/papersdynamapsUsability.pdf , 2001.[13] J. Nielsen, “Finding Usability Problems through Heuristic Evaluation,” Proc. ACM Int'l Conf. Human-Computer Interaction (CHI '92), pp 373-380, 1992.[14] J.D. Mackinlay, “Automating the Design of Graphical Presentations of Relational Information,” ACM Trans. Graphics, vol. 5, pp.110-141, 1986.[15] E. Tufte, The Visual Display of Quantitative Information. Graphics Press, 1983.[16] C. Freitas, P. Luzzardi, R. Cava, M. Pimenta, A. Winckler, and L. Nedel, “Evaluating Usability of Information Visualization Techniques,” Proc. Advanced Visual Interfaces Conf. (AVI '02), pp. 373-374, 2002.[17] B. Shneiderman, “The Eyes Have It: A Task by Data Type Taxonomy,” Proc. IEEE Symp. Visual Languages '96, pp. 336-343, 1996.[18] C. Ware, Information Visualization: Perception for Design. Morgan Kaufmann, 2004.[19] W.S. Cleveland, The Elements of Graphing Data. Hobart Press, 1994.[20] O. Juarez, “CAEVA: Cognitive Architecture to Evaluate Visualization Applications,” Proc. Int'l Conf. Information Visualization (IV '03), pp. 589-595, 2003.[21] V. Gonzales and A. Kobsa, “A Workplace Study of the Adoption of Information Visualization Systems,” Proc. I-KNOW '03: Third Int'l Conf. Knowledge Management, pp 92-102, 2003.[22] J. Rieman, “A Field Study of Exploratory Learning Strategies,” ACM Trans. Computer-Human Interaction, vol. 3, pp. 189-218, 1996.[23] J. Seo and B. Shneiderman, “Knowledge Discovery in High Dimensional Data: Case Studies and a User Survey for the Rank-by-Feature Framework,” IEEE Trans. Visualization and Computer Graphics, vol. 12, no. 3, May/June 2006.[24] H. Causton, J. Quackenbush, and A. Brazma, A Beginners Guide to Microarray Data Analysis. Blackwell Publishing, 2003.[25] D. Berrar, W. Dubitzky, and M Granzow, A Practical Approach to Microarray Data Analysis. Kluwer Academic Publishers, 2004.[26] Cambridge Healthtech Inst. Data Visualization and Interpretation, Making Breakthroughs Possible in the Omics Research, http://www.healthtech.com/2002dvs/, 2002.[27] Cambridge Healthtech Inst. Data Visualization and Interpretation, Deciphering the Data Deluge, http://www.healthtech.com/2003/mdeindex.asp , 2003.[28] B.M. Bolstad, R.A. Irizarry, M. Astrand, and T.P. Speed, “A Comparison of Normalization Methods for High Density Oligonucleotide Array Data Based on Bias and Variance,” Bioinformatics, vol. 19, pp. 185-193, 2003.[29] W. Pan, “A Comparative Review of Statistical Methods for Discovering Differentially Expressed Genes in Replicated Microarray Experiments,” Bioinformatics, vol. 18, no. 4, pp. 546-554, 2002.[30] M.B. Eisen, P.T. Spellman, P.O. Brown, and D. Botstein, “Cluster Analysis and Display of Genome-Wide Expression Patterns,” Proc. Nat'l Academy of Science US, vol. 95, pp. 14863-14868, 1998.[31] Y. Leung, “Functional Genomics,” http:/genomicshome.com, 2004.[32] N. Bolshakova, Microarray Software Catalogue, http://www.cs.tcd.ie/Nadia.Bolshakovasoftwaretotal.html , 2004.[33] GeneSifter, http://www.genesifter.netweb/, 2006.[34] N.E. Olson, “Identification of Cell Cycle Genes Regulated during Erythroid Differentiation,” http://www.microarraysuccess.orgweb/, 2005.[35] R. Stevens, C. Goble, P. Baker, and A. Brass, “A Classification of Tasks in Bioinformatics,” Bioinformatics, vol. 17, no. 2, pp 180-188, 2001.[36] V.L. O'Day, A. Adler, A. Kuchinsky, and A. Bouch, “When Worlds Collide: Molecular Biology as Interdisciplinary Collaboration,” Proc. European Conf. Computer Supported Cooperative Work, 2001.[37] W.E. Mackay, G. Pothier, C. Letondal, K. Bøegh, and H.E. Sørensen, “The Missing Link: Augmenting Biology Laboratory Notebooks,” Proc. Symp. User Interface Software and Technology, pp.41-50, 2002.[38] P. Saraiya, C. North, and K. Duca, “Visualization of Biological Pathways: Requirements Analysis, Systems Evaluation, and Research Agenda,” Information Visualization, vol. 4, no. 3, 2005.[39] J. Thomas and K. Cook, Illuminating the Path: The Research and Development Agenda for Visual Analytics, IEEE Press, http://nvac.pnl.govagenda.stm, 2006.[40] Spotfire® Decisionsite for Functional Genomics, http:/www.Spotfire.com, 2006.[41] PathwayAssist® Ariadne Ge nomics, 2006, http://www.ariad negenomics.com/products pathway.html.[42] GenMapp, Gene Map Annotator and Pathway Profiler, 2006, www.genmapp.org.[43] Q Value Software, http://faculty.washington.edu/jstoreyqvalue /, 2006.[44] KalaidaGraph, http:/www.synergy.com/, 2006.[45] Affymetrix, http://www.affymetrix.comindex.affx, 2006.[46] The R Project for Statistical Computing, http:/www.r-project. org/, 2006.[47] J. Storey and R. Tibshirani, “Statistical Significance for Genomewide Studies,” Proc. Nat'l Academy of Sciences, vol. 100, no. 16, pp.9440-9445, 2003.
Index Terms:
Evaluation/methodology, Graphical User Interface (GUI), information visualization, visualization systems and software, visualization and methodologies.
Citation:
Purvi Saraiya, Chris North, Vy Lam, Karen A. Duca, "An Insight-Based Longitudinal Study of Visual Analytics," IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 6, pp. 1511-1522, Nov./Dec. 2006, doi:10.1109/TVCG.2006.85
