A Scalable Parallel Formulation of the Backpropagation Algorithm for Hypercubes and Related Architectures
Issue No. 10 - October (1994 vol. 5)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/71.313123
<p>We present a new technique for mapping the backpropagation algorithm on hypercubeand related architectures. A key component of this technique is a network partitioningscheme called checkerboarding. Checkerboarding allows us to replace the all-to-allbroadcast operation performed by the commonly used vertical network partitioningscheme, with operations that are much faster on the hypercubes and relatedarchitectures. Checkerboarding can be combined with the pattern partitioning techniqueto form a hybrid scheme that performs better than either one of these schemes.Theoretical analysis and experimental results on nCUBE and CM5 show that our schemeperforms better than the other schemes, for both uniform and nonuniform networks.</p>
Index Termsbackpropagation; hypercube networks; neural nets; parallel architectures; parallelmachines; parallel algorithms; scalable parallel formulation; backpropagation algorithm;hypercubes; network partitioning scheme; checkerboarding; all-to-all broadcastoperation; vertical network partitioning scheme; pattern partitioning technique; hybridscheme; performance evaluation; nCUBE; CM5; nonuniform networks; uniform networks; neural networks
S. Shekhar, V. Kumar and M. Amin, "A Scalable Parallel Formulation of the Backpropagation Algorithm for Hypercubes and Related Architectures," in IEEE Transactions on Parallel & Distributed Systems, vol. 5, no. , pp. 1073-1090, 1994.