The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2006 vol.17)
pp: 193-204
Marianne Winslett , IEEE Computer Society
ABSTRACT
<p><b>Abstract</b>—Scientific applications often need to write out large arrays and associated metadata periodically for visualization or restart purposes. In this paper, we present <it>active buffering</it>, a high-level transparent buffering scheme for collective I/O, in which processors actively organize their idle memory into a hierarchy of buffers for periodic output data. It utilizes idle memory on the processors, yet makes no assumption regarding runtime memory availability. Active buffering can perform background I/O while the computation is going on, is extensible to remote I/O for more efficient data migration, and can be implemented in a portable style in today's parallel I/O libraries. It can also mask performance problems of scientific data formats used by many scientists. Performance experiments with both synthetic benchmarks and real simulation codes on multiple platforms show that active buffering can greatly reduce the visible I/O cost from the application's point of view.</p>
INDEX TERMS
Parallel I/O library design, performance optimization, experimentation.
CITATION
Xiaosong Ma, Jonghyun Lee, Marianne Winslett, "High-Level Buffering for Hiding Periodic Output Cost in Scientific Simulations", IEEE Transactions on Parallel & Distributed Systems, vol.17, no. 3, pp. 193-204, March 2006, doi:10.1109/TPDS.2006.36
28 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool