This Article 
 Bibliographic References 
 Add to: 
Learning Bias in Neural Networks and an Approach to Controlling Its Effect in Monotonic Classification
September 1993 (vol. 15 no. 9)
pp. 962-966

As a learning machine, a neural network using the backpropagation training algorithm is subject to learning bias. This results in unpredictability of boundary generation behavior in pattern recognition applications, especially in the case of small training sample size. It is suggested that in a large class of pattern recognition problems such as managerial and other problems possessing monotonicity properties, the effect of learning bias can be controlled by using multiarchitecture monotonic function neural networks.

[1] E. Barnard and D. Casasent, "A comparison between criterion functions for linear classifiers, with an application to neural nets,"IEEE Trans. Syst. Man Cybern., vol. 19, no. 5, pp. 1030-1041, 1989.
[2] T. Kohonen, G. Barna, and R. Chrisley, "Statistical pattern recognition with neural networks: Benchmarking studies," inProc. IEEE Int. Conf. Neural Networks, (San Diego), 1988, pp. I:61-68.
[3] D. F. Specht, "Probabilistic neural networks,"Neural Networks, vol. 3, no. 1, pp. 109-118, 1990.
[4] D. F. Specht, "Probabilistic neural networks and the polynomial adaline as complementary techniques for classification,"IEEE Trans. Neural Networks, vol. 1, no. 1, pp. 111-121, 1990.
[5] T. Kawabata, "Generalization effects of k-neighbor interpolation training,"Neural Comput., vol. 3, pp. 409-417, 1991.
[6] D. P. Casasent and E. Barnard, "Adaptive-clustering optical neural net,"Applied Optics, vol. 29, pp. 2603-2615, 1990.
[7] D.E. Rumelhart and D. McClelland, eds.,Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vols. 1-2, MIT Press, Cambridge, Mass., 1986.
[8] D. J. Hand,Discrimination and Classification. New York: Wiley, 1981.
[9] F. F. Soulie, P. Gallinari, Y. L. Cun, and S. Thiria, "Evaluation of network architectures on test learning tasks," inProc. IEEE First Int. Conf. Neural Networks(San Diego), 1987, pp. II:653-660.
[10] P. E. Utgoff, "Shift of bias for inductive concept learning," inMachine Learning: An Artificial Intelligence Approach(R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds.). Los Altos, CA: Morgan Kaufman 1986, pp. 107-148, vol. II.
[11] L. Rendell, R. Seshu, and D. Tcheng, "More robust concept learning using dynamically-variable bias," inProc. Fourth Int. Workshop Machine Learning(M. B. Morgan, Ed.). Los Altos, CA: Morgan Kaufmann, pp. 66-78, 1987.
[12] G. Cybenko, "Approximation by superpositions of a sigmoidal function,"Math. Contr. Signals Syst., vol. 2, no. 4, pp. 303-314, 1989.
[13] P. E. Utgoff,Machine Learning Of Inductive Bias. Boston, MA: Kluwer, 1986.
[14] E. I. Altman, "Financial ratios, discriminant analysis and the prediction of corporate bankruptcy,"J. Finance, vol. 28, pp. 589-609, 1968.
[15] C. C. Greer, "Deciding to accept or reject a marginal retail credit application,"J. Retailing, vol. 43, pp. 44-53, 1968.
[16] V. Ramanujam, N. Venkatraman, and J. C. Camillus, "Multiobjective assessment of effectiveness of strategic planning: A discriminant analysis approach,"Acad Management J., vol. 29, pp. 347-372, 1986.
[17] R. H. Keeney and H. Raiffa,Decisions with Multiple Objectives: Preferences and Value Tradeoffs. New York: Wiley, 1976.
[18] R. A. Fisher, "The use of multiple measurements in taxonomic problems,"Ann. Eugenics, vol. 7, pp. 179-188, 1936.
[19] Y. T. Chien and T. J. Killeen, "Computer and statistical considerations for oil spill identification," inHandbook of Statistics, (P. R. Krishnaiah and L. N. Kanel, Eds.). New York: North Holland, 1982, pp. 651-671, vol. 2.
[20] N. P. Archer and S. Wang, "Application of the back propagation neural network algorithm with monotonicity constraints for two-group classification problems,"Decision Sci., vol. 24, pp. 60-75, 1993.
[21] A. Wieland and R. Leighton, "Geometric analysis of neural network capabilities," inProc. IEEE First Int. Conf. Neural Network(San Diego) 1987, pp. III:385-392.

Index Terms:
boundary generation unpredictability; monotonic classification; backpropagation training algorithm; learning bias; pattern recognition; multiarchitecture monotonic function neural networks; backpropagation; learning (artificial intelligence); neural nets; pattern recognition
N.P. Archer, S. Wang, "Learning Bias in Neural Networks and an Approach to Controlling Its Effect in Monotonic Classification," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 9, pp. 962-966, Sept. 1993, doi:10.1109/34.232084
Usage of this product signifies your acceptance of the Terms of Use.