This Article 
 Bibliographic References 
 Add to: 
Three-Dimensional Structured Networks for Matrix Equation Solving
December 1991 (vol. 40 no. 12)
pp. 1337-1346

Two three-dimensional structured networks are developed for solving linear equations and the Lyapunov equation. The basic idea of the structured network approaches is to first represent a given equation-solving problem by a 3-D structured network so that if the network matches a desired pattern array, the weights of the linear neurons give the solution to the problem: then, train the 3-D structured network to match the desired pattern array using some training algorithms; and finally, obtain the solution to the specific problem from the converged weights of the network. The training algorithms for the two 3-D structured networks are proved to converge exponentially fast to the correct solutions. Simulations were performed to show the detailed convergence behaviors of the 3-D structured networks.

[1] D. P. Bertsekas and J. N. Tsitsiklis,Parallel and Distributed Computations. Englewood Cliffs, NJ: Prentice-Hall, 1989.
[2] D. Heller, "A survey of parallel algorithms in numerical linear algebra,"SIAM Rev., vol. 20, pp. 740-777, 1978.
[3] A. H. Sameh and D. J. Kuck, "On stable parallel linear solver,"J. ACM, vol. 25, pp. 81-91, 1978.
[4] S. L. Johnson, "Solving tridiagonal systems on ensemble architectures,"SIAM J. Sci. Stat. Comput., vol. 8, pp. 345-392, 1987.
[5] R. E. Lord, J. S. Kowalik, and S. P. Kumar, "Solving linear algebraic expressions on an MIMO computer,"J. ACM, vol. 30, pp. 103-117, 1983.
[6] A. Bojanczyk, R. P. Brent, and H. T. Kung, "Numerically stable solution of linear equations using mesh-connected processors,"SIAM J. Sci. Stat. Comput., vol. 5, pp. 95-104, 1984.
[7] S.Y. Kung,VLSI Array Processors, Prentice Hall, Englewood Cliffs, N.J. 1988.
[8] H. T. Kung, B. Sproul, and G. Steele,VLSI Systems and Computations. Rockville, MD: Computer Science Press, 1981.
[9] L. Csanky, "Fast parallel matrix inversion algorithms,"SIAM J. Comput., vol. 5, pp. 618-623, 1976.
[10] A. Bojanczy, "Complexity of solving linear systems in different models of computation,"J. ACM, vol. 32, pp. 792-803, 1984.
[11] S. Lo, B. Phillipe, and A. Sameh, "A multiprocessor algorithm for the symmetric tridiagonal eigenvalue problem,"SIAM J. Sci. Stat. Comput., vol. 8, pp. s155-s165, 1987.
[12] J. Dongarra and D. Sorensen, "A fully parallel algorithm for the symmetric eigenvalue problem,"SIAM J. Sci. Stat. Comput., vol. 8, pp. s139-s154, 1987.
[13] R. Barlow, D. Evans, and J. Shanehchi, "Parallel multisection applied to the eigenvalue problem,"The Comput. J., vol. 26, pp. 6-9, 1983.
[14] E. Jessup and D. Sorensen, "A multiprocessor scheme for the singular value decomposition," inParallel Processing for Scientific Computing, 1989, pp. 61-66.
[15] E. Jessup, "Parallel solution of the symmetric tridiagonal eigenproblem," Res. Rep. YALEU/DCS/RR-728, 1989.
[16] A. Brenneret al., "Panel: How do we make parallel processing a reality?: Bridging the gap between theory and practice," inProc. 5th Int. Parallel Processing Symp., 1991, pp. 648-653.
[17] N. Morgan, Ed.,Artificial Neural Networks Electronic Implementations. IEEE Computer Society Press, 1990.
[18] U. Ramacher and U. Ruckert, Eds.,VLSI Design of Neural Networks. Norwell, MA: Kluwer Academic, 1991.
[19] L. X. Wang and J. M. Mendel, "Structured trainable networks for matrix algebra," inProc. 1990 Int. Joint Conf. Neural Networks, vol. 2, 1990, pp. II125-II132.
[20] L. X. Wang and J. M. Mendel, "Matrix computations and equation solving using structured networks and training," inProc. IEEE 1990 Conf. Decision and Control, 1990, pp. 1747-1750.
[21] L. X. Wang and J. M. Mendel, "Parallel structured networks for solving a wide variety of matrix algebra problems,"J. Parallel Distributed Processing, Mar. 1992, to be published.
[22] M. M. Polycarpou and P. A. Ioannou, "Learning and convergence analysis of neural-type structured networks,"IEEE Trans. Neural Networks, 1991, to be published.
[23] L. X. Wang and J. M. Mendel, "Cumulant-based parameter estimation using structured networks,"IEEE Trans. Neural Networks, vol. 2, no. 1, pp. 73-83, 1991.
[24] P. Werbos, "New tools for predictions and analysis in the behavioral science," Ph.D. dissertation, Harvard Univ. Committee on Applied Mathematics, 1974.
[25] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representation by error propagation,"Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vols. 1 and 2. Cambridge, MA: MIT Press, 1986.
[26] D. B. Parker, "Learning logic," Tech. Rep. TR-47, M.I.T. Center for Computational Economics and Statistics, 1985.
[27] B. Widrow and S. D. Stearns,Adaptive Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1985.
[28] D. G. Luenberger, Optimization byVector Space Methods. New York: Wiley, 1969.
[29] J. W. Brewer, "Kronecker products and matrix calculus in system theory,"IEEE Trans. Circuits Syst., vol. CAS-25, pp. 772-781, 1978.

Index Terms:
three dimensional structured networks; matrix equation solving; linear equations; Lyapunov equation; equation-solving problem; pattern array; linear neurons; training algorithms; Lyapunov methods; matrix algebra; neural nets.
L.-X. Wang, J.M. Mendel, "Three-Dimensional Structured Networks for Matrix Equation Solving," IEEE Transactions on Computers, vol. 40, no. 12, pp. 1337-1346, Dec. 1991, doi:10.1109/12.106219
Usage of this product signifies your acceptance of the Terms of Use.