The Community for Technology Leaders
RSS Icon
Issue No.12 - Dec. (2012 vol.18)
pp: 2779-2788
Jagoda Walny , University of Calgary
Bongshin Lee , Microsoft Research
Paul Johns , Microsoft Research
Nathalie Henry Riche , Microsoft Research
Sheelagh Carpendale , University of Calgary
Current interfaces for common information visualizations such as bar graphs, line graphs, and scatterplots usually make use of the WIMP (Windows, Icons, Menus and a Pointer) interface paradigm with its frequently discussed problems of multiple levels of indirection via cascading menus, dialog boxes, and control panels. Recent advances in interface capabilities such as the availability of pen and touch interaction challenge us to re-think this and investigate more direct access to both the visualizations and the data they portray. We conducted a Wizard of Oz study to explore applying pen and touch interaction to the creation of information visualization interfaces on interactive whiteboards without implementing a plethora of recognizers. Our wizard acted as a robust and flexible pen and touch recognizer, giving participants maximum freedom in how they interacted with the system. Based on our qualitative analysis of the interactions our participants used, we discuss our insights about pen and touch interactions in the context of learnability and the interplay between pen and touch gestures. We conclude with suggestions for designing pen and touch enabled interactive visualization interfaces.
Context awareness, Sociology, Statistical analysis, Data visualization, Writing, data exploration, Pen and touch, interaction, Wizard of Oz, whiteboard
Jagoda Walny, Bongshin Lee, Paul Johns, Nathalie Henry Riche, Sheelagh Carpendale, "Understanding Pen and Touch Interaction for Data Exploration on Interactive Whiteboards", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 12, pp. 2779-2788, Dec. 2012, doi:10.1109/TVCG.2012.275
[1] M. Bostock and J. Heer, Protovis: A graphical toolkit for visualization IEEE TVCG (InfoVis 2009), 15, 6, pp. 1121-1128, 2009.
[2] J.N. Bott, J. LaViola, and J. Joseph, A pen-based tool for visualizing vector mathematics Proc. SBIM 2010, pp. 103-110, 2010.
[3] P. Brandl, C. Forlines, D. Wigdor., M. Haller, and C. Shen, Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces Proc. AVI 2008, pp. 154-161, 2008.
[4] S. Branham, G. Golovchinsky, S. Carter,, and J.T. Biehl., Let's go from the whiteboard: Supporting transitions in work through whiteboard capture and reuse. Proc. CHI 2010, pp. 75-84, 2010.
[5] J. Browne, B. Lee, S. Carpendale., N. Riche, and T. Sherwood, Data analysis on interactive whiteboards through sketch-based interaction Proc. ITS 2011, pp. 154-157, 2011.
[6] W.O. Chao, T. Munzner, and M. van, de Panne. Poster: Rapid pen­ centric authoring of improvisational visualizations with Napkin Vis. Posters Compendium Info Vis 2010, 2010.
[7] Cintiq 21UX Pen Display │ Wacom A mericas. (accessed June 26, 2012).
[8] N. Dahlbäck, A. Jönsson, and L. Ahrenberg, Wizard of Oz studies — why and how Knowledge-Based Systems, 6, pp. 258-266, 1993.
[9] R.C. Davis, T.S. Saponas, M. Shilman,, and J. A. Landay,SketchWizard: Wizard of Oz prototyping of pen-based user interfaces, Proc. UIST 2007, pp. 119-128, 2007.
[10] T. Dwyer, B. Lee, D. Fisher., K. Inkpen, P. Isenberg., G. Robertson, and C. North, Understanding multi-touch manipulation for surface computing IEEE TVCG (InfoVis 2009), 25, 19, pp. 961-968, 2009.
[11] N. Elmqvist,A. Vande Moere, H.-C. Jetter, D. Cernea., H. Reiterer, and T.J. Jankun-Kelly., Fluid interaction for information visualization. Information Visualization, 10, pp. 327-340, 2011.
[12] Fast Analytics and Rapid-fire Business Intelligence from Tableau Software │ Tableau Software. http:/ (accessed June 26, 2012).
[13] M. Frisch, J. Heydekorn, and R. Dachselt, Investigating multi-touch and pen gestures for diagram editing on interactive surfaces Proc. ITS 2009, pp. 149-156, 2009.
[14] M.L. Gick and K.J. Holyoak., Schema induction and analogical transfer. Cognitive Psychology, 15, pp. 1-38, 1983.
[15] L. Grammel, M. Tory, and M.A. Storey, How information visualization novices construct visualizations IEEE TVCG (Info Vis 2010), 16, 6, pp. 943-952, 2010.
[16] K. Hinckley, K. Yatani, M. Pahud., N. Coddington, J. Rodenhouse., A. Wilson, H. Benko,, and B. Buxton., Pen + touch = new tools. Proc. UIST 2010, pp. 27-36, 2010.
[17] U. Hinrichs and M.S.T. Carpendale., Gestures in the wild: Studying multi-touch gesture sequences on interactive tabletop exhibits. Proc. CHI 2011, pp. 3023-3032, 2011.
[18] C. Holz and S. Feiner, Relaxed selection techniques for querying time-series graphs UIST 2009, pp. 213-222, 2009.
[19] IBM SPSS software for predictive analytics. spss (accessed June 26, 2012).
[20] P. Isenberg and S. Carpendale, Interactive tree comparison for co-located collaborative information visualization IEEE TVCG (InfoVis 2007), 13, 6, pp. 1232-1239, 2007.
[21] P. Isenberg, U. Hinrichs, M. Hancock,, and S. Carpendale., “Digital ables for Collaborative Information Exploration.” Tabletops - Horizontal Interactive Displays, C. Miiller-Tomfelde, ed., Springer-Verlag, pp. 387-405, 2010.
[22] B. Jordan, and A. Henderson, Interaction analysis: Foundations and practice J Learning Sciences, 4, pp. 39-103, 1995.
[23] J. La Viola, J. Joseph, and R.C. Zeleznik, MathPad2: A system for the creation and exploration of mathematical sketches ACM SIGGRAPH 2007 Courses, 2007.
[24] J. Leitner, C. Rendl, F. Perteneder., A. Gokcezade, T. Seifried., M. Haller, R. Zeleznik,, and A. Bragdon., NiCE formula editor. Proc. ACM SIGGRAPH 2010 Talks. PP. 55:1-55:1. 2010.
[25] Luidia Inc.: eBeam Classic Whiteboard: eBeam Whiteboard Turns Any Whiteboard into a Digital Copyboard. http://www.e-beam.comlproductsebeam-classic-whiteboards.html (accessed June 26, 2012).
[26] M.R. Morris, J.O. Wobbrock, and A.D. Wilson, Understanding users’ preferences for surface gestures Proc. GI 2010, pp. 261-268, 2010.
[27] Microsoft Excel - Spreadsheet - Office. com. (accessed June 26, 2012).
[28] E.D. Mynatt, The writing on the wall Proc. Interact 1999, pp. 196-204, 1999.
[29] C. Müller- Tomfelde., Tabletops - Horizontal Interactive Displays, Springer-Verlag, 2010.
[30] L. Olsen, F.F. Samavati, M.C. Sousa,, and J. A. Jorge., Sketch-based modeling: A survey, Computers & Graphics, 33, pp. 85-103, 2009.
[31] J. Nielsen., Noncommand user interfaces. CACM, 36, 4, pp. 83-99, 1993.
[32] F.F. Samavati, L. Olsen, and J.A. Jorge., Sketch-based Interfaces and Modeling, pp. 1-15, 2011.
[33] K. Ryall, N. Lesh, T. Lanning., D. Leigh, H. Miyashita,, and S. Makino., QueryLines: Approximate query for visual browsing. Ext. Abst. CHI 2005, pp. 1765-1768, 2005.
[34] E. Saund and E. Lank., “Minimizing Modes for Smart Selection in Sketching/Drawing Interfaces.” Sketch-based Interfaces and Modeling, J. Jorge, and F. Samavati, ed., Springer-Verlag, pp. 55-80, 2011.
[35] P. Saulnier, E. Sharlin, and S. Greenberg, Exploring interruption in HRI using Wizard of Oz Proc. HRI 2010, pp. 125-126, 2010.
[36] B. Shneiderman, Direct manipulation: A step beyond programming Iancuazes IEEE Computer. 16. 8.00.57–69. 1983.
[37] S. Schmidt, M. Nacenta, R. Dachselt,, and S. Carpendale., A set of multitouch graph interaction techniques. Proc. ITS 2011, pp. 113-116, 2011.
[38] I.E. Sutherland, Sketchpad: A man-machine graphical communication system Proc. AFIPS Sprine Joint Como. Conf. pp. 329-346. 1963.
[39] R. Stewart and M.C. Schraefel, Graph sketcher: Extending illustration to quantitative zraohs Proc. CHI 2009.00. 1113–1116. 2009.
[40] R. Taib and N. Ruiz, Wizard of Oz for multimodal interfaces design: Deployment considerations Proc. HCI 2007, pp. 232-241, 2007.
[41] A. Teiche, A.K. Rai, C. Yanc., C. Moore, D. Solms., G. Cetin, J. Riggio., N. Ramseyer, P. Dintino,, and L. Muller., Multi-touch technologies. NUI Group, 2009.
[42] United Nations,Department of Economic and Social Affairs, Population Division. World Population Prospects: The 2010 Revision, CD-ROM Edition_ 2011.
[43] S. Voida, M. Podlaseck, R. Kjeldsen, and C. Pinhanez., A study on the manipulation of 2D objects in a projector/camera-based augmented reality environment. Proc. CHI 2005, pp. 611-620, 2005.
[44] J. Walny, S. Carpendale, N. Henry Riche, G. Venolia, and P. Fawcett, Visual thinking in action: Visualizations as used on white boards IEEE TVCG (Info Vis 2011), 16, 6, pp. 2508-2517, 2011.
[45] M. Wattenberg, Sketching a graph to query a time-series database Ext. Abs. CHI 2001, pp. 381-382, 2001.
[46] D. Wigdor and D. Wixon., Brave NUI World: Designing Natural User Interfaces for Touch and Gesture. Morgan Kaufmann, 2011.
[47] J.O. Wobbrock, M.R. Morris, and A.D. Wilson, User-defined gestures for surface computing Proc. CHI 2009, pp. 1083-1092, 2009.
21 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool