The Community for Technology Leaders
2013 IEEE International Conference on Cluster Computing (CLUSTER) (2013)
Indianapolis, IN, USA
Sept. 23, 2013 to Sept. 27, 2013
ISBN: 978-1-4799-0898-1
pp: 1-8
Hongbo Zou , College of Computing, Georgia Institute of Technology, Atlanta, USA
Karsten Schwan , College of Computing, Georgia Institute of Technology, Atlanta, USA
Magdalena Slawinska , College of Computing, Georgia Institute of Technology, Atlanta, USA
Matt Wolf , College of Computing, Georgia Institute of Technology, Atlanta, USA
Greg Eisenhauer , College of Computing, Georgia Institute of Technology, Atlanta, USA
Fang Zheng , College of Computing, Georgia Institute of Technology, Atlanta, USA
Jai Dayal , College of Computing, Georgia Institute of Technology, Atlanta, USA
Jeremy Logan , Scientific Data Group, Oak Ridge National Laboratory, TN, USA
Qing Liu , Scientific Data Group, Oak Ridge National Laboratory, TN, USA
Scott Klasky , Scientific Data Group, Oak Ridge National Laboratory, TN, USA
Tanja Bode , Center for Relativistic Astrophysics, Georgia Institute of Technology, Atlanta, USA
Michael Clark , Center for Relativistic Astrophysics, Georgia Institute of Technology, Atlanta, USA
Matt Kinsey , Center for Relativistic Astrophysics, Georgia Institute of Technology, Atlanta, USA
ABSTRACT
The remote visual exploration of live data generated by scientific simulations is useful for scientific discovery, performance monitoring, and online validation for the simulation results. Online visualization methods are challenged, however, by the continued growth in the volume of simulation output data that has to be transferred from its source - the simulation running on the high end machine - to where it is analyzed, visualized, and displayed. A specific challenge in this context is limits in the communication bandwidth between data source(s) and sinks. Previous work places queries ‘near’ data sources, exploiting their data reduction capabilities, but such work does not address the common scenario in which scientists make multiple different queries on the data being produced. This paper considers the general case in which science users are interested in different (sub)sets of the data produced by a high end simulation. We offer the FlexQuery online data query system that can deploy and execute data queries ‘along’ the I/O and analytics pipelines. FlexQuery carefully extends such analytics pipelines, using online performance monitoring and data location tracking, to realize data queries in ways that minimize additional data movement and offer low latency in data query execution. Using a real-world scientific application - the Maya astrophysics code and its analytics workflow - we demonstrate FlexQuery's ability to dynamically deploy queries for low-latency remote data visualization.
INDEX TERMS
data reduction, remote visualization, online query
CITATION

H. Zou et al., "FlexQuery: An online query system for interactive remote visual data exploration at large scale," 2013 IEEE International Conference on Cluster Computing (CLUSTER), Indianapolis, IN, USA USA, 2014, pp. 1-8.
doi:10.1109/CLUSTER.2013.6702635
94 ms
(Ver 3.3 (11022016))