The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - Dec. (2013 vol.24)
pp: 2451-2461
Snaider Carrillo , University of Ulster, Londonderry
Jim Harkin , University of Ulster, Londonderry
Liam J. McDaid , University of Ulster, Londonderry
Fearghal Morgan , National University of Ireland, Galway
Sandeep Pande , National University of Ireland, Galway
Seamus Cawley , National University of Ireland, Galway
Brian McGinley , National University of Ireland, Galway
ABSTRACT
Spiking neural networks (SNNs) attempt to emulate information processing in the mammalian brain based on massively parallel arrays of neurons that communicate via spike events. SNNs offer the possibility to implement embedded neuromorphic circuits, with high parallelism and low power consumption compared to the traditional von Neumann computer paradigms. Nevertheless, the lack of modularity and poor connectivity shown by traditional neuron interconnect implementations based on shared bus topologies is prohibiting scalable hardware implementations of SNNs. This paper presents a novel hierarchical network-on-chip (H-NoC) architecture for SNN hardware, which aims to address the scalability issue by creating a modular array of clusters of neurons using a hierarchical structure of low and high-level routers. The proposed H-NoC architecture incorporates a spike traffic compression technique to exploit SNN traffic patterns and locality between neurons, thus reducing traffic overhead and improving throughput on the network. In addition, adaptive routing capabilities between clusters balance local and global traffic loads to sustain throughput under bursting activity. Analytical results show the scalability of the proposed H-NoC approach under different scenarios, while simulation and synthesis analysis using 65-nm CMOS technology demonstrate high-throughput, low-cost area, and power consumption per cluster, respectively.
INDEX TERMS
Neural networks, Computer architecture, Microprocessors, On chip architectures, Network topology,spiking neural networks, Interconnection architecture, network-on-chip, neurocomputers, real-time distributed
CITATION
Snaider Carrillo, Jim Harkin, Liam J. McDaid, Fearghal Morgan, Sandeep Pande, Seamus Cawley, Brian McGinley, "Scalable Hierarchical Network-on-Chip Architecture for Spiking Neural Network Hardware Implementations", IEEE Transactions on Parallel & Distributed Systems, vol.24, no. 12, pp. 2451-2461, Dec. 2013, doi:10.1109/TPDS.2012.289
REFERENCES
[1] W. Gerstner, Spiking Neuron Models: Single Neurons, Populations, Plasticity, first ed. Cambridge Univ. Press, 2002.
[2] K.A. Zaghloul and K. Boahen, "A Silicon Retina That Reproduces Signals in the Optic Nerve." J. Neural Eng., vol. 3, no. 4, pp. 257-67, Dec. 2006.
[3] S. Cawley, F. Morgan, B. McGinley, S. Pande, L. McDaid, S. Carrillo, and J. Harkin, "Hardware Spiking Neural Network Prototyping and Application," Genetic Programming and Evolvable Machines, vol. 12, pp. 257-280, 2011.
[4] H. Paugam-Moisy and S. Bohte, "Computing with Spiking Neuron Networks," Handbook of Natural Computing, G. Rozenberg, T. Back, and J. Kok, eds., pp. 1-47, Springer, 2009.
[5] H. de Garis, C. Shuo, B. Goertzel, and L. Ruiting, "A World Survey of Artificial Brain Projects, Part I: Large-Scale Brain Simulations," Neurocomputing, vol. 74, nos. 1-3, pp. 3-29, 2010.
[6] T. Trappenberg, Fundamentals of Computational Neuroscience, second ed. Oxford Univ. Press, 2009.
[7] S. Furber and S. Temple, "Neural Systems Engineering," J. Royal Soc. Interface, vol. 4, no. 13, pp. 193-206, 2007.
[8] A. Hemani, A. Jantsch, S. Kumar, A. Postula, J. Öberg, M. Millberg, and D. Lindqvist, "Network on Chip: An Architecture for Billion Transistor Era," Proc. IEEE NorChip Conf., pp. 166-173, 2000.
[9] L. Benini and G. De Micheli, "Networks on Chips: A New Soc Paradigm," Computer, vol. 35, no. 1, pp. 70-78, Jan. 2002.
[10] W. Dally and B. Towles, "Route Packets, Not Wires: On-Chip Interconnection Networks," Proc. 38th Ann. Design Automation Conf., pp. 684-689, 2001.
[11] T. Theocharides, G. Link, N. Vijaykrishnan, M. Invin, and V. Srikantam, "A Generic Reconfigurable Neural Network Architecture as a Network on Chip," Proc. IEEE Int'l SOC Conf., pp. 191-194, Sept. 2004.
[12] R. Emery, A. Yakovlev, and G. Chester, "Connection-Centric Network for Spiking Neural Networks," Proc. Third ACM/IEEE Int'l Symp. Networks-on-Chip, pp. 144-152, May 2009.
[13] J. Harkin, F. Morgan, L. McDaid, S. Hall, B. McGinley, and S. Cawley, "A Reconfigurable and Biologically Inspired Paradigm for Computation Using Network-on-Chip and Spiking Neural Networks," Int'l J. Reconfigurable Computing, vol. 2009, pp. 1-13, 2009.
[14] S. Carrillo, J. Harkin, L. McDaid, S. Pande, S. Cawley, B. McGinley, and F. Morgan, "Advancing Interconnect Density for Spiking Neural Network Hardware Implementations Using Traffic-Aware Adaptive Network-on-Chip Routers," Neural Networks, vol. 33, pp. 42-57, Sept. 2012.
[15] G. Indiveri, B. Linares-Barranco, T.J. Hamilton, A. van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P. Häfliger, S. Renaud, J. Schemmel, G. Cauwenberghs, J. Arthur, K. Hynna, F. Folowosele, S. Saighi, T. Serrano-Gotarredona, J. Wijekoon, Y. Wang, and K. Boahen, "Neuromorphic Silicon Neuron Circuits," Frontiers in Neuroscience, vol. 5, p. 73, Jan. 2011.
[16] S.H. Strogatz, "Exploring Complex Networks," Nature, vol. 410, no. 6825, pp. 268-76, Mar. 2001.
[17] D.J. Watts and S.H. Strogatz, "Collective Dynamics of 'Small-World' Networks," Nature, vol. 393, no. 6684, pp. 440-2, June 1998.
[18] D.S. Bassett, D.L. Greenfield, A. Meyer-Lindenberg, D.R. Weinberger, S.W. Moore, and E.T. Bullmore, "Efficient Physical Embedding of Topologically Complex Information Processing Networks in Brains and Computer Circuits." PLoS Computational Biology, vol. 6, no. 4, p. e1000748, Apr. 2010.
[19] S. Rodrigo, J. Flich, A. Roca, S. Medardoni, D. Bertozzi, J. Camacho, F. Silla, and J. Duato, "Cost-Efficient on-Chip Routing Implementations for CMP and MPSoC Systems," IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, vol. 30, no. 4, pp. 534-547, Apr. 2011.
[20] R. Das, S. Eachempati, A.K. Mishra, V. Narayanan, and C.R. Das, "Design and Evaluation of a Hierarchical On-Chip Interconnect for Next-Generation CMPs," Proc. IEEE 15th Int'l Symp. High Performance Computer Architecture, pp. 175-186, Feb. 2009.
[21] J.-Y. Kim, J. Park, S. Lee, M. Kim, J. Oh, and H.-J. Yoo, "A 118.4 GB/s Multi-Casting Network-on-Chip with Hierarchical Star-Ring Combined Topology for Real-Time Object Recognition," IEEE J. Solid-State Circuits, vol. 45, no. 7, pp. 1399-1409, July 2010.
[22] D. Vainbrand and R. Ginosar, "Scalable Network-On-Chip Architecture for Configurable Neural Networks," Microprocessors and Microsystems, vol. 35, no. 2, pp. 152-166, Mar. 2011.
[23] S. Carrillo, J. Harkin, L. McDaid, S. Pande, S. Cawley, B. McGinley, and F. Morgan, "Hierarchical Network-on-Chip and Traffic Compression for Spiking Neural Network Implementations," Proc. IEEE/ACM Sixth Int'l Symp. Networks-on-Chip, pp. 83-90, May 2012.
[24] J. Misra and I. Saha, "Artificial Neural Networks in Hardware: A Survey of Two Decades of Progress," Neurocomputing, vol. 74, nos. 1-3, pp. 239-255, 2010.
[25] H. Markram, "The Blue Brain Project," Nature Rev. Neuroscience, vol. 7, no. 2, pp. 153-159, 2006.
[26] M. Blumrich, D. Chen, P. Coteus, A. Gara, and M. Giampapa, "Design and Analysis of the Bluegene/L Torus Interconnection Network," IBM Research Report RC23025 (W0312-022), vol. 23025, 2003.
[27] "The Neurogrid Website," http://www.stanford.edu/group/brainsinsilicon neurogrid.html, 2009.
[28] S. Choudhary, S. Sloan, S. Fok, A. Neckar, E. Trautmann, P. Gao, T. Stewart, C. Eliasmith, and K. Boahen, "Silicon Neurons That Compute," Proc. Int'l Conf. Artificial Neural Networks (ICANN '12), 2012.
[29] J. Schemmel, J. Fieres, and K. Meier, "Wafer-Scale Integration of Analog Neural Networks," Proc. IEEE Int'l Joint Conf. Neural Networks, pp. 431-438, June 2008.
[30] D. Brüderle, M.A. Petrovici, B. Vogginger, M. Ehrlich, T. Pfeil, S. Millner, A. Grübl, K. Wendt, E. Müller, M.-O. Schwartz, D.H. de Oliveira, S. Jeltsch, J. Fieres, M. Schilling, P. Müller, O. Breitwieser, V. Petkov, L. Muller, A.P. Davison, P. Krishnamurthy, J. Kremkow, M. Lundqvist, E. Muller, J. Partzsch, S. Scholze, L. Zühl, C. Mayr, A. Destexhe, M. Diesmann, T.C. Potjans, A. Lansner, R. Schüffny, J. Schemmel, and K. Meier, "A Comprehensive Workflow for General-Purpose Neural Modeling with Highly Configurable Neuromorphic Hardware Systems," Biological Cybernetics, vol. 104, nos. 4/5, pp. 263-96, May 2011.
[31] J. Schemmel, J. Fieres, and K. Meier, "Wafer-Scale Integration of Analog Neural Networks," Proc. IEEE Int'l Joint Conf. Neural Networks, pp. 431-438, 2008.
[32] "The BrainScaleS Website," http:/brainscales.kip.uni-heidelberg. de /, 2011.
[33] "The Spinnaker Project Website," http://apt.cs.man.ac.uk/ projectsSpiNNaker /, 2007.
[34] S. Furber and A. Brown, "Biologically-Inspired Massively-Parallel Architectures—Computing beyond a Million Processors," Proc. Ninth Int'l Conf. Application of Concurrency to System Design, pp. 3-12, July 2009.
[35] S.B. Furber, D.R. Lester, L.A. Plana, J.D. Garside, E. Painkras, S. Temple, and A.D. Brown, "Overview of the SpiNNaker System Architecture," IEEE Trans. Computers, pp. 1-1, 2012.
[36] "The SyNAPSE Project Website," http://www.almaden.ibm. com/cs/peopledmodha /, 2011.
[37] J.-s. Seo, B. Brezzo, Y. Liu, B.D. Parker, S.K. Esser, R.K. Montoye, B. Rajendran, J.A. Tierno, L. Chang, D.S. Modha, and D.J. Friedman, "A 45 nm CMOS Neuromorphic Chip with a Scalable Architecture for Learning in Networks of Spiking Neurons," Proc. IEEE Custom Integrated Circuits Conf. (CICC), pp. 1-4, Sept. 2011.
[38] L. McDaid, S. Hall, and P. Kelly, "A Programmable Facilitating Synapse Device," Proc. IEEE Int'l Joint Conf. Neural Networks, pp. 1615-1620, 2008.
[39] S. Carrillo, J. Harkin, L. McDaid, S. Pande, and F. Morgan, "An Efficient, High-Throughput Adaptive NoC Router for Large Scale Spiking Neural Network Hardware Implementations," Proc. Ninth Int'l Conf. Evolvable Systems: From Biology to Hardware, pp. 133-144, 2010.
[40] S. Carrillo, J. Harkin, L. McDaid, S. Pande, S. Cawley, and F. Morgan, "Adaptive Routing Strategies for Large Scale Spiking Neural Network Hardware Implementations," Proc. 21th Int'l Conf. Artificial Neural Networks, pp. 77-84, 2011.
35 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool