This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Validating a Biometric Authentication System: Sample Size Requirements
December 2006 (vol. 28 no. 12)
pp. 1902-1319
Authentication systems based on biometric features (e.g., fingerprint impressions, iris scans, human face images, etc.) are increasingly gaining widespread use and popularity. Often, vendors and owners of these commercial biometric systems claim impressive performance that is estimated based on some proprietary data. In such situations, there is a need to independently validate the claimed performance levels. System performance is typically evaluated by collecting biometric templates from n different subjects, and for convenience, acquiring multiple instances of the biometric for each of the n subjects. Very little work has been done in 1) constructing confidence regions based on the ROC curve for validating the claimed performance levels and 2) determining the required number of biometric samples needed to establish confidence regions of prespecified width for the ROC curve. To simplify the analysis that address these two problems, several previous studies have assumed that multiple acquisitions of the biometric entity are statistically independent. This assumption is too restrictive and is generally not valid. We have developed a validation technique based on multivariate copula models for correlated biometric acquisitions. Based on the same model, we also determine the minimum number of samples required to achieve confidence bands of desired width for the ROC curve. We illustrate the estimation of the confidence bands as well as the required number of biometric samples using a fingerprint matching system that is applied on samples collected from a small population.

[1] J.R. Beveridge, K. She, and B.A. Draper, “A Nonparametric Statistical Comparison of Principal Component and Linear Discriminant Subspaces for Face Recognition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, Dec. 2001.
[2] R. Bolle, J. Connell, S. Pankanti, N. Ratha, and A. Senior, Guide to Biometrics. Springer, 2004.
[3] R. Bolle, N. Ratha, and S. Pankanti, “Error Analysis of Pattern Recognition Systems: The Subsets Bootstrap,” Computer Vision and Image Understanding, vol. 93, no. 1, pp. 1-33, Jan. 2004.
[4] R.M. Bolle, S. Pankanti, and N.K. Ratha, “Evaluation Techniques for Biometrics-Based Authentication Systems (FRR),” Proc. 14th Int'l Conf. Pattern Recognition, pp. 2831-2837, Aug. 2000.
[5] U. Cherubini, E. Luciano, and W. Vecchiato, Copula Methods in Finance. Wiley, 2004.
[6] S.C. Dass, Y. Zhu, and A.K. Jain, “Validating a Biometric Authentication System: Sample Size Requirements,” Technical Report MSU-CSE-05-23, Dept. of Computer Science and Eng., Michigan State Univ., East Lansing, Aug. 2005.
[7] A.K. Jain, L. Hong, and R. Bolle, “On-Line Fingerprint Verification,” IEEE Trans. Pattern Recognition and Machine Intelligence, vol. 19, no. 4 pp. 302-314, 1997.
[8] A.K. Jain, S. Prabhakar, and A. Ross, “Fingerprint Matching: Data Acquisition and Performance Evaluation,” Technical Report TR99- 14, Michigan State Univ., East Lansing, 1999.
[9] R.A. Johnson and D.W. Wichern, Applied Multivariate Statistical Analysis. Prentice Hall, 1988.
[10] R.J. Micheals and T.E. Boult, “Efficient Evaluation of Classification and Recognition Systems,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, Dec. 2001.
[11] R.G. Miller, Simultaneous Statistical Inference. Springer-Verlag, 1981.
[12] D.F. Morrison, Multivariate Statistical Methods. McGraw-Hill, 1990.
[13] R.B. Nelsen, An Introduction to Copulas. Springer, 1999.
[14] J. Porter, “On the "30 Error Criterion,” Nat'l Biometric Center Collected Works, J. Wayman, ed., pp. 51-56, 2000.
[15] C.R. Rao, Linear Statistical Inference and Its Applications. Wiley, 1991.
[16] M.E. Schuckers, “Using the Beta-Binomial Distribution to Assess Performance of a Biometric Identification Device,” Int'l J. Image and Graphics, special issue on biometrics, vol. 3, no. 3, pp. 523-529, July 2003.
[17] “Best Practices in Testing and Reporting Performance of Biometric Devices,” U.K. Biometrics Working Group, www.cesg.gov.uk/technologybiometrics, 2000.
[18] J. Wayman, “Technical Testing and Evaluation of Biometric Identification Devices,” Biometrics: Personal Identification in Networked Society, A.K. Jain, R. Bolle, and S. Pankanti, eds. Kluwer Academic, 1999.
[19] J. Wayman, “Confidence Interval and Test Size Estimation for Biometric Data,” Nat'l Biometric Center Collected Works, J. Wayman, ed., pp. 91-95, 2000.
[20] J. Wayman, “Technical Testing and Evaluation of Biometric Identification Devices,” Nat'l Biometric Center Collected Works, J. Wayman, ed., pp. 67-89, 2000.

Index Terms:
Biometric authentication, error estimation, Gaussian copula models, bootstrap, ROC confidence bands.
Citation:
Sarat C. Dass, Yongfang Zhu, Anil K. Jain, "Validating a Biometric Authentication System: Sample Size Requirements," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 1902-1319, Dec. 2006, doi:10.1109/TPAMI.2006.255
Usage of this product signifies your acceptance of the Terms of Use.