Cluster Computing and the Grid, IEEE International Symposium on (2010)
Melbourne, VIC, Australia
May 17, 2010 to May 20, 2010
In high performance computing (HPC), the applications are periodically check pointed to stable storage to increase the success rate of long executions. Nowadays, the overhead imposed by disk-based checkpoint is about 20% of execution time and in the next years it will be more than 50% if the checkpoint frequency increases as the fault frequency increases. Diskless checkpoint has been introduced as a solution to avoid the IO bottleneck of disk-based checkpoint. However, the encoding time, the dedicated resources (the spares) and the memory overhead imposed by diskless checkpoint are significant obstacles against its adoption. In this work, we address these three limitations: 1) we propose a fault tolerant model able to tolerate up to 50% of process failures with a low check pointing overhead 2) our fault tolerance model works without spare node, while still guarantying high reliability, 3) we use solid state drives to significantly increase the checkpoint performance and avoid the memory overhead of classic diskless checkpoint.
Fault tolerance, Diskless checkpoint, Erasure codes, HPC, Petascale systems
Satoshi Matsuoka, Leonardo Arturo Bautista Gomez, Naoya Maruyama, Franck Cappello, "Distributed Diskless Checkpoint for Large Scale Systems", Cluster Computing and the Grid, IEEE International Symposium on, vol. 00, no. , pp. 63-72, 2010, doi:10.1109/CCGRID.2010.40