2nd New Zealand Two-Stream International Conference on Artificial Neural Networks and Expert Systems (ANNES '95) Indicators of hidden neuron functionality: the weight matrix versus neuron behaviour Dunedin, New Zealand November 20-November 23 ISBN: 0-8186-7174-2
Pruning of redundant or less important hidden neurons from the popular backpropagation trained neural networks is useful for a host of reasons, ranging from improvements of generalisation performance, to use as a precursor for rule extraction. For pruning it is necessary to identify hidden neurons with similar functionality. We have previously used a pruning process based on the behaviour of the hidden neurons in an image processing application to produce a quality driven compression by eliminating the least different hidden neurons. We consider the computationally cheaper alternative using only the trained weight matrix of the neural networks at each stage of the compression process. We conclude that the weight matrix is not sufficient for differentiating the functionality of the hidden neurons for this task, being essentially the functional equivalence problem which is computationally intractable.
Index Terms:
feedforward neural nets; backpropagation; image coding; data compression; hidden neuron functionality; weight matrix; neuron behaviour; hidden neuron pruning; backpropagation trained neural networks; generalisation performance; rule extraction; pruning process; image processing application; quality driven compression; least different hidden neurons; trained weight matrix; compression process; functional equivalence problem
Citation:
T.D. Gedeon, "Indicators of hidden neuron functionality: the weight matrix versus neuron behaviour," annes, pp.26, 2nd New Zealand Two-Stream International Conference on Artificial Neural Networks and Expert Systems (ANNES '95), 1995 Usage of this product signifies your acceptance of the Terms of Use. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||