Subscribe

Issue No.07 - July (2012 vol.23)

pp: 1255-1266

Vahid Jalili-Marandi , OPAL-RT Technologies Inc., Montreal

Zhiyin Zhou , University of Alberta, Edmonton

Venkata Dinavahi , University of Alberta, Edmonton

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPDS.2011.291

ABSTRACT

This paper proposes large-scale transient stability simulation based on the massively parallel architecture of multiple graphics processing units (GPUs). A robust and efficient instantaneous relaxation (IR)-based parallel processing technique which features implicit integration, full Newton iteration, and sparse LU-based linear solver is used to run the multiple GPUs simultaneously. This implementation highlights the combination of coarse-grained algorithm-level parallelism with fine-grained data-parallelism of the GPUs to accelerate large-scale transient stability simulation. Multithreaded parallel programming makes the entire implementation highly transparent, scalable, and efficient. Several large test systems are used for the simulation with a maximum size of 9,984 buses and 2,560 synchronous generators all modeled in detail resulting in matrices that are larger than 20\hbox{,}000\times 20\hbox{,}000.

INDEX TERMS

Graphics processors, instantaneous relaxation, large-scale systems, multiple GPUs, Newton-Raphson method, parallel multithreaded programming, power system simulation, power system transient stability, sparse direct solvers.

CITATION

Vahid Jalili-Marandi, Zhiyin Zhou, Venkata Dinavahi, "Large-Scale Transient Stability Simulation of Electrical Power Systems on Parallel GPUs",

*IEEE Transactions on Parallel & Distributed Systems*, vol.23, no. 7, pp. 1255-1266, July 2012, doi:10.1109/TPDS.2011.291REFERENCES

- [1] IEEE/CIGRE Joint Task Force on Stability Terms and Definitions, "Definition and Classification of Power System Stability,"
IEEE Trans. Power Systems, vol. 19, no. 2, pp. 1387-1401, May 2004.- [2] K. Morison, L. Wang, and P. Kundur, "Power System Security Assessment,"
IEEE Power and Energy Magazine, vol. 2, no. 5, pp. 30-39, Sept./Oct. 2004.- [3] L. Wang and K. Morison, "Implementation of Online Security Assessment,"
IEEE Power and Energy Magazine, vol. 4, no. 5, pp. 46-59, Sept./Oct. 2006.- [4] EPRI TR-104352, "Analytical Methods for Contingency Selection and Ranking for Dynamic Security Analysis,"
Power and Energy Soc. General Meeting, Conversion and Delivery of Electrical Energy in the 21st Century, Project 3103-03 Final Report, Sept. 1994.- [5] D. Blythe, "Rise of the Graphics Processor,"
Proc. IEEE, vol. 96, no. 5, pp. 761-778, May 2008.- [6] J.D. Owens, M. Houston, D. Luebke, S. Green, J.E. Stone, and J.C. Phillips, "GPU Computing,"
Proc. IEEE, vol. 96, no. 5, pp. 879-899, May 2008.- [7] W. Liu, B. Schmidt, G. Voss, and W. Muller-Wittig, "Streaming Algorithms for Biological Sequence Alignment on GPUs,"
IEEE Trans. Parallel and Distributed Systems, vol. 18, no. 9, pp. 1270-1281, Sept. 2007.- [8] R. Weber, A. Gothandaraman, R.J. Hinde, and G.D. Peterson, "Comparing Hardware Accelerators in Scientific Applications: A Case Study,"
IEEE Trans. Parallel and Distributed Systems, vol. 22, no. 1, pp. 1045-9219, Jan. 2011.- [9] W.J. van der Laan, A.C. Jalba, and J.B.T.M. Roerdink, "Accelerating Wavelet Lifting on Graphics Hardware Using CUDA,"
IEEE Trans. Parallel and Distributed Systems, vol. 22, no. 1, pp. 132-146, Jan. 2011.- [10] NVIDIA CUDA: Programming Guide, June 2008.
- [11] T.D. Han and T.S. Abdelrahman, "hiCUDA: High-Level GPGPU Programming,"
IEEE Trans. Parallel and Distributed Systems, vol. 22, no. 1, pp. 78-90, Jan. 2011.- [12] V. Jalili-Marandi and V. Dinavahi, "SIMD-Based Large-Scale Transient Stability Simulation on the Graphics Processing Unit,"
IEEE Trans. Power Systems, vol. 25, no. 3, pp. 1589-1599, Aug. 2010.- [13] V. Jalili-Marandi and V. Dinavahi, "Instantaneous Relaxation Based Real-Time Transient Stability Simulation,"
IEEE Trans. Power Systems, vol. 24, no. 3, pp. 1327-1336, Aug. 2009.- [14] B. Stott, "Power System Dynamic Response Calculations,"
Proc. IEEE, vol. 67, no. 2, pp. 219-241, Feb. 1979.- [15] F.F. de Mello, J.W. Felts, T.F. Laskowski, and L.J. Opple, "Simulating Fast and Slow Dynamic Effects in Power Systems,"
IEEE Computer Applications in Power, vol. 5, no. 3, pp. 33-38, July 1992.- [16] H.W. Dommel and N. Sato, "Fast Transient Stability Solutions,"
IEEE Trans. Power Apparatus and Systems, vol. PAS-91, no. 4, pp. 1643-1650, July 1972.- [17] W. Gear, "Simultaneous Numerical Solutions of Differential-Algebraic Equations,"
IEEE Trans. Circuit Theory, vol. CT-18, no. 1, pp. 89-95, Jan. 1971.- [18] L. Elden and L. Wittmeyer-Koch,
Numerical Analysis—An Introduction. Academic Press Inc., 1990.- [19] G. Kron,
Diakoptics-Piecewise Solutions of Large Systems, vols. 158/162, General Electric and Also Published by McDonald, 1963.- [20] M. La Scala, G. Sblendorio, A. Bose, and J.Q. Wu, "Comparison of Algorithms for Transient Stability Simulations on Shared and Distributed Memory Multiprocessors,"
IEEE Trans. Power Systems, vol. 11, no. 4, pp. 2045-2050, Nov. 1996.- [21] F.L. Alvarado, "Parallel Solution of Transient Problems by Trapezoidal Integration,"
IEEE Trans. Power Apparatus and Systems, vol. PAS-98, no. 3, pp. 1080-1090, May/June 1979.- [22] M. La Scala, R. Sbrizzai, and F. Torelli, "A Pipelined-in-Time Parallel Algorithm for Transient Stability Analysis,"
IEEE Trans. Power Systems, vol. 6, no. 2, pp. 715-722, May 1991.- [23] M.L. Crow and M. Ilic, "The Parallel Implementation of the Waveform Relaxation Method for Transient Stability Simulations,"
IEEE Trans. Power Systems, vol. 5, no. 3, pp. 922-932, Aug. 1990.- [24] M.J. Flynn, "Very High Speed Computing Systems,"
Proc. IEEE, vol. 54, no. 12, pp. 1901-1909, Dec. 1966.- [25] J.R. Gurd, "A Taxonomy of Parallel Computer Architectures,"
Proc. Int'l Specialist Seminar Design and Application of Parallel Digital Processors, pp. 57-61, Apr. 1988.- [26] H.H. Happ, C. Pottle, and K.A. Wirgau, "An Assessment of Computer Technology for Large Scale Power System Simulation,"
Proc. IEEE Conf. Power Industry Computer Applications, pp. 316-324, May 1979.- [27] F.M. Brasch, J.E. Van Ness, and S.C. Kang, "Simulation of a Multiprocessor Network for Power System Problems,"
IEEE Trans. Power Apparatus and Systems, vol. PAS-101, no. 2, pp. 295-301, Feb. 1982.- [28] S.Y. Lee, H.D. Chiang, K.G. Lee, and B.Y. Ku, "Parallel Power System Transient Stability Analysis on Hypercube Multiprocessors,"
IEEE Trans. Power Systems, vol. 6, no. 3, pp. 1337-1343, Aug. 1991.- [29] H. Taoka, S. Abe, and S. Takeda, "Fast Transient Stability Solution Using an Array Processor,"
IEEE Trans. Power Apparatus and Systems, vol. PAS-102, no. 12, pp. 3835-3841, Dec. 1983.- [30] M. Takatoo, S. Abe, T. Bando, K. Hirasawa, M. Goto, T. Kato, and T. Kanke, "Floating Vector Processor for Power System Simulation,"
IEEE Trans. Power Apparatus and Systems, vol. PAS-104, no. 12, pp. 3360-3366, Dec. 1985.- [31] P. Forsyth, R. Kuffel, R. Wierckx, J. Choo, Y. Yoon, and T. Kim, "Comparison of Transient Stability Analysis and Large-Scale Real Time Digital Simulation,"
Proc. IEEE Porto Power Tech, vol. 4, pp. 1-7, Sept. 2001.- [32] J.S. Chai and A. Bose, "Bottlenecks in Parallel Algorithms for Power System Stability Analysis,"
IEEE Trans. Power Systems, vol. 8, no. 1, pp. 9-15, Feb. 1993.- [33] H. You, V. Vittal, and X. Wang, "Slow Coherency-Based Islanding,"
IEEE Trans. Power Systems, vol. 19, no. 1, pp. 483-491, Feb. 2004.- [34] NVIDIA, "Specification: Tesla S1070 GPU Computing System," Oct. 2008.
- [35] E. Lindholm, J. Nickolls, S. Oberman, and J. Montrym, "NVIDIA Tesla: A Unified Graphics and Computing Architecture,"
IEEE Micro, vol. 28, no. 2, pp. 39-55, Mar./Apr. 2008.- [36] Visual Studio Developer Center, http://msdn.microsoft.com/ en-uslibrary /, 2012.
- [37] L. Buatois, G. Caumon, and B. Lévy, "Concurrent Number Cruncher: A GPU Implementation of a General Sparse Linear Solver,"
Int'l J. Parallel, Emergent and Distributed Systems, vol. 24, no. 3, pp. 205-223, June 2009.- [38] M. Fatica, "Accelerating Linpack with CUDA on Heterogenous Clusters,"
Proc. Second Workshop General Purpose Processing on Graphics Processing Units, pp. 46-51, Mar. 2009.- [39] S. Tomov, R. Nath, H. Ltaief, and J. Dongarra, "Dense Linear Algebra Solvers for Multicore with GPU Accelerators,"
Proc. IEEE Symp. Parallel and Distributed Processing (IPDPS '10), pp. 1-8, Jan. 2010.- [40] G.P. Krawezik and G. Poole, "Accelerating the ANSYS Direct Sparse Solver with GPUs,"
Proc. Symp. Application Accelerators in High Performance Computing (SAAHPC '10), 2010.- [41] R.F. Lucas, G. Wagenbreth, D.M. Davis, and R. Grimes, "Multifrontal Computations on GPUs and Their Multi-Core Hosts,"
Proc. Ninth Int'l Conf. High Performance Computing for Computational Science (VECPAR '10), vol. 6449, pp. 71-82, 2011.- [42] CULA, http:/www.culatools.com/, 2012.
- [43] J.R. Gilbert, C. Moler, and R. Schreiber, "Sparse Matrices in Matlab: Design and Implementation,"
SIAM J. Matrix Analysis and Applications, vol. 13, pp. 333-356, 1992.- [44] M. Garland, "Sparse Matrix Computations on Manycore GPU's,"
Proc. 45th Ann. Design Automation Conf., pp. 2-6, June 2008.- [45] NVIDIA, "CUDA CUBLAS Library," Mar. 2008.
- [46] IEEE Task Force on Computer and Analytical Methods, "Parallel Processing in Power Systems Computation,"
IEEE Trans. Power Systems, vol. 7, no. 2, pp. 629-638, May 1992.- [47] P. Kundur,
Power System Stability and Control. McGraw-Hill, 1994. |