The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.06 - June (2012 vol.18)
pp: 852-864
John Biddiscombe , Swiss National Supercomputing Centre, Manno
Jerome Soumagne , Swiss National Supercomputing Centre, Manno
Guillaume Oger , INRIA Bordeaux Sud-Ouest, Talence
David Guibert , Ecole Centrale de Nantes, Nantes
Jean-Guillaume Piccinali , Swiss National Supercomputing Centre, Manno
ABSTRACT
Interfacing a GUI driven visualization/analysis package to an HPC application enables a supercomputer to be used as an interactive instrument. We achieve this by replacing the IO layer in the HDF5 library with a custom driver which transfers data in parallel between simulation and analysis. Our implementation using ParaView as the interface, allows a flexible combination of parallel simulation, concurrent parallel analysis, and GUI client, either on the same or separate machines. Each MPI job may use different core counts or hardware configurations, allowing fine tuning of the amount of resources dedicated to each part of the workload. By making use of a distributed shared memory file, one may read data from the simulation, modify it using ParaView pipelines, write it back, to be reused by the simulation (or vice versa). This allows not only simple parameter changes, but complete remeshing of grids, or operations involving regeneration of field values over the entire domain. To avoid the problem of manually customizing the GUI for each application that is to be steered, we make use of XML templates that describe outputs from the simulation (and inputs back to it) to automatically generate GUI controls for manipulation of the simulation.
INDEX TERMS
Parallel I/O, distributed/network graphics, software libraries.
CITATION
John Biddiscombe, Jerome Soumagne, Guillaume Oger, David Guibert, Jean-Guillaume Piccinali, "Parallel Computational Steering for HPC Applications Using HDF5 Files in Distributed Shared Memory", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 6, pp. 852-864, June 2012, doi:10.1109/TVCG.2012.63
REFERENCES
[1] A. Henderson, ParaView Guide, A Parallel Visualization Application. Kitware, Inc., http:/www.paraview.org, 2005.
[2] J. Biddiscombe, J. Soumagne, G. Oger, D. Guibert, and J.-G. Piccinali, “Parallel Computational Steering and Analysis for HPC Applications Using a ParaView Interface and the HDF5 DSM Virtual File Driver,” Proc. Eurographics Symp. Parallel Graphics and Visualization, T. Kuhlen, R. Pajarola and K. Zhou eds., pp. 91-100, 2011.
[3] J. Brooke, P. Coveney, J. Harting, S. Jha, S. Pickles, R. Pinning, and A. Porter, “Computational Steering in RealityGrid,” Proc. UK E-Science All Hands Meeting, pp. 885-888, 2003.
[4] N. Richart, A. Esnard, and O. Coulaud, “Toward a Computational Steering Environment for Legacy Coupled Simulations,” Proc. Sixth Int'l Symp. Parallel and Distributed Computing (ISPDC), p. 43, July 2007.
[5] H. Childs, E. Brugger, K. Bonnell, J. Meredith, M. Miller, B. Whitlock, and N. Max, “A Contract Based System for Large Data Visualization,” Proc. IEEE Visualization (VIS '05), pp. 191-198, Oct. 2005.
[6] B. Whitlock, J.M. Favre, and J.S. Meredith, “Parallel in Situ Coupling of Simulation with a Fully Featured Visualization System,” Proc. Eurographics Symp. Parallel Graphics and Visualization, T. Kuhlen, R. Pajarola and K. Zhou eds., pp. 101-109, 2011.
[7] N. Fabian, K. Moreland, D. Thompson, A.C. Bauer, P. Marion, B. Geveci, M. Rasquin, and K.E. Jansen, “The ParaView Coprocessing Library: A Scalable, General Purpose in Situ Visualization Library,” Proc. IEEE Symp. Large-Scale Data Analysis and Visualization (LDAV), Oct. 2011.
[8] C. Docan, M. Parashar, and S. Klasky, “DataSpaces: An Interaction and Coordination Framework for Coupled Simulation Workflows,” Proc. 19th ACM Int'l Symp. High Performance Distributed Computing (HPDC), pp. 25-36, 2010.
[9] J. Lofstead, F. Zheng, S. Klasky, and K. Schwan, “Adaptable, Metadata Rich IO Methods for Portable High Performance IO,” Proc. IEEE Int'l Symp. Parallel and Distributed Processing (IPDPS), pp. 1-10, 2009.
[10] F. Zhang, C. Docan, M. Parashar, and S. Klasky, “Enabling Multi-Physics Coupled Simulations within the PGAS Programming Framework,” Proc. IEEE/ACM 11th Int'l Symp. Cluster, Cloud and Grid Computing (CCGrid), pp. 84-93, 2011.
[11] V. Vishwanath, M. Hereld, V. Morozov, and M.E. Papka, “Topology-Aware Data Movement and Staging for I/O Acceleration on Blue Gene/P Supercomputing Systems,” Proc. Int'l Conf. High Performance Computing, Networking, Storage and Analysis (SC), pp. 19:1-19:11, 2011.
[12] J. Soumagne, J. Biddiscombe, and J. Clarke, “An HDF5 MPI Virtual File Driver for Parallel in-Situ Post-Processing,” Proc. 17th European MPI Users' Group Meeting Conf. Recent Advances in the Message Passing Interface, R. Keller, E. Gabriel, M. Resch, and J. Dongarra eds., pp. 62-71, 2010.
[13] J. Soumagne and J. Biddiscombe, “Computational Steering and Parallel Online Monitoring Using RMA through the HDF5 DSM Virtual File Driver,” Proc. Int'l Conf. Computational Science (ICCS), vol. 4, pp. 479-488, 2011.
[14] W. Gropp and R. Thakur, “Revealing the Performance of MPI RMA Implementations,” Proc. 14th European PVM/MPI Users' Group Meeting Recent Advances in Parallel Virtual Machine and Message Passing Interface, F. Cappello T. Herault and J. Dongarra eds., pp. 272-280, 2007.
[15] J. Soumagne, J. Biddiscombe, and A. Esnard, “Data Redistribution Using One-Sided Transfers to in-Memory HDF5 Files,” Proc. 18th European MPI Users' Group Meeting Conf. Recent Advances in the Message Passing Interface, A. Danalis, D. Nikolopoulos and J. Dongarra eds., pp. 198-207, 2011.
[16] A. Cedilnik, B. Geveci, K. Moreland, J. Ahrens, and J. Favre, “Remote Large Data Visualization in the ParaView Framework,” Proc. Eurographics Symp. Parallel Graphics and Visualization, B. Raffin, A. Heirich, and L.P. Santos eds., pp. 163-170, 2006.
[17] J.A. Clarke and E.R. Mark, “Enhancements to the eXtensible Data Model and Format (XDMF),” Proc. DoD High Performance Computing Modernization Program Users Group Conf. (HPCMP-UGC '07), pp. 322-327, 2007.
[18] P. Goswami, P. Schlegel, B. Solenthaler, and R. Pajarola, “Interactive SPH Simulation and Rendering on the GPU,” Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation (SCA '10), pp. 55-64, 2010.
[19] G. Oger, E. Jacquin, P.-M. Guilcher, L. Brosset, J.-B. Deuff, D.L. Touze, and B. Alessandrini, “Simulations of Complex Hydro-Elastic Problems Using the Parallel Sph Model SPH-Flow,” Proc. Fourth SPHERIC Int'l Worksop, 2009.
26 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool