This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Performance Evaluation of Fingerprint Verification Systems
January 2006 (vol. 28 no. 1)
pp. 3-18
ASCII Text
x 
 
Raffaele Cappelli, Dario Maio, Davide Maltoni, James L. Wayman, Anil K. Jain, "Performance Evaluation of Fingerprint Verification Systems," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 3-18, January, 2006.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2006.20,
author = {Raffaele Cappelli and Dario Maio and Davide Maltoni and James L. Wayman and Anil K. Jain},
title = {Performance Evaluation of Fingerprint Verification Systems},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {28},
number = {1},
issn = {0162-8828},
year = {2006},
pages = {3-18},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2006.20},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Performance Evaluation of Fingerprint Verification Systems
IS - 1
SN - 0162-8828
SP3
EP18
EPD - 3-18
A1 - Raffaele Cappelli,
A1 - Dario Maio,
A1 - Davide Maltoni,
A1 - James L. Wayman,
A1 - Anil K. Jain,
PY - 2006
KW - Index Terms- Biometric systems
KW - fingerprint verification
KW - performance evaluation
KW - technology evaluation
KW - FVC.
VL - 28
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
This paper is concerned with the performance evaluation of fingerprint verification systems. After an initial classification of biometric testing initiatives, we explore both the theoretical and practical issues related to performance evaluation by presenting the outcome of the recent Fingerprint Verification Competition (FVC2004). FVC2004 was organized by the authors of this work for the purpose of assessing the state-of-the-art in this challenging pattern recognition application and making available a new common benchmark for an unambiguous comparison of fingerprint-based biometric systems. FVC2004 is an independent, strongly supervised evaluation performed at the evaluators' site on evaluators' hardware. This allowed the test to be completely controlled and the computation times of different algorithms to be fairly compared. The experience and feedback received from previous, similar competitions (FVC2000 and FVC2002) allowed us to improve the organization and methodology of FVC2004 and to capture the attention of a significantly higher number of academic and commercial organizations (67 algorithms were submitted for FVC2004). A new, "Light” competition category was included to estimate the loss of matching performance caused by imposing computational constraints. This paper discusses data collection and testing protocols, and includes a detailed analysis of the results. We introduce a simple but effective method for comparing algorithms at the score level, allowing us to isolate difficult cases (images) and to study error correlations and algorithm "fusion.” The huge amount of information obtained, including a structured classification of the submitted algorithms on the basis of their features, makes it possible to better understand how current fingerprint recognition systems work and to delineate useful research directions for the future.

[1] R. Cappelli, “Synthetic Fingerprint Generation,” Handbook of Fingerprint Recognition, D. Maltoni, D. Maio, A.K. Jain, and S. Prabhakar, eds. New York: Springer, 2003.
[2] R. Cappelli, A. Erol, D. Maio, and D. Maltoni, “Synthetic Fingerprint-Image Generation,” Proc. 15th Int'l Conf. Pattern Recognition, pp. 475-478, Sept. 2000.
[3] R. Cappelli, D. Maio, and D. Maltoni, “Modelling Plastic Distortion in Fingerprint Images,” Proc. Second Int'l Conf. Advances in Pattern Recognition, pp. 369-376, Mar. 2001.
[4] R. Cappelli, D. Maio, and D. Maltoni, “Synthetic Fingerprint-Database Generation,” Proc. 16th Int'l Conf. Pattern Recognition, vol. 3, pp. 744-747, Aug. 2002.
[5] Y. Dit-Yan et al., “SVC2004: First International Signature Verification Competition,” Proc. Int'l Conf. Biometric Authentication, pp. 16-22, July 2004.
[6] G. Doddington et al., “Sheep, Goats, Lambs and Wolves: A Statistical Analysis of Speaker Performance,” Proc. Int'l Conf. Language and Speech Processing, pp. 1351-1354, Nov. 1998.
[7] J. Fierrez-Aguilar, L. Nanni, J. Ortega-Garcia, R. Cappelli, and D. Maltoni, “Combining Multiple Matchers for Fingerprint Verification: A Case Study in FVC2004,” Proc. 13th Int'l Conf. Image Analysis and Processing, Sept. 2005.
[8] C. Wilson et al., “Fingerprint Vendor Technology Evaluation 2003: Summary of Results and Analysis Report,” NISTIR 7123, Nat'l Inst. of Standards and Technology, http:/fpvte.nist.gov, June 2004.
[9] D.M. Blackburn, J.M. Bone, and P.J. Phillips, “Facial Recognition Vendor Test 2000 Evaluation Report,” http://www.frvt.orgFRVT2000, Feb. 2001.
[10] P.J. Phillips, P. Grother, R.J Micheals, D.M. Blackburn, E Tabassi, and J.M. Bone, “Facial Recognition Vendor Test 2002 Evaluation Report,” http://www.frvt.orgFRVT2002, Mar. 2003.
[11] Fingerprint Verification Competition (FVC2000), http://bias.csr.unibo.itfvc2000, 2000.
[12] Fingerprint Verification Competition (FVC2002), http://bias.csr.unibo.itfvc2002, 2002.
[13] Fingerprint Verification Competition (FVC2004), http://bias.csr.unibo.itfvc2004, 2004.
[14] ISO/IEC JTC1 SC 37 WG 3 Final Committee Draft 19794-2: Finger Minutiae Pattern Format, 2004.
[15] A.K. Jain, S. Prabhakar, and S. Chen, “Combining Multiple Matchers for a High Security Fingerprint Verification System,” Pattern Recognition Letters, vol. 20, nos. 11-13, pp. 1371-1379, 1999.
[16] A.K. Jain and A. Ross, “Fingerprint Mosaicking,” Proc. Int'l Conf. Acoustic Speech and Signal Processing, vol. 4, pp. 4064-4067, 2002.
[17] D. Maio, D. Maltoni, R. Cappelli, J.L. Wayman, and A.K. Jain, “FVC2000: Fingerprint Verification Competition,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 24, no. 3, pp. 402-412, Mar. 2002.
[18] D. Maio, D. Maltoni, R. Cappelli, J.L. Wayman, and A.K. Jain, “FVC2002: Second Fingerprint Verification Competition,” Proc. 16th Int'l Conf. Pattern Recognition, vol. 3, pp. 811-814, Aug. 2002.
[19] D. Maio, D. Maltoni, R. Cappelli, J.L. Wayman, and A.K. Jain, “FVC2004: Third Fingerprint Verification Competition,” Proc. Int'l Conf. Biometric Authentication, pp. 1-7, July 2004.
[20] D. Maltoni, D. Maio, A.K. Jain, and S. Prabhakar, Handbook of Fingerprint Recognition. New York: Springer, 2003.
[21] A. Mansfield, G. Kelly, D. Chandler, and J. Kane, “Biometric Product Testing Final Report,” Issue 1.0, U.K. Nat'l Physical Lab, Mar. 2001.
[22] A. Martin, M. Przybocki, and J. Campbell, “The NIST Speaker Recognition Evaluation Program,” Biometric Systems Technology, Design and Performance Evaluation, J. Wayman, A. Jain, D. Maltoni, and D. Maio, eds. London: Springer-Verlag, 2004.
[23] J. Matasetal, “Comparison of Face Verification Results on the XM2VTS Database,” Proc. 15th Int'l Conf. Pattern Recognition, vol. 4, pp. 858-863, Sept. 2000.
[24] K. Messeretal, “Face Authentication Competition on the BANCA Database,” Proc. Int'l Conf. Biometric Authentication, pp. 8-15, July 2004.
[25] Nat'l Inst. of Standards and Technology Speaker Recognition Evaluation, http://www.nist.gov/speech/tests/spkindex.htm , 2004.
[26] S.B. Panetal, “An Ultra-Low Memory Fingerprint Matching Algorithm and Its Implementation on a 32-Bit Smart Card,” IEEE Trans. Consumer Electronics, vol. 49, no. 2, pp. 453-459, May 2003.
[27] S. Pankanti, N.K. Ratha, and R.M. Bolle, “Structure in Errors: A Case Study in Fingerprint Verification,” Proc. 16th Int'l Conf. Pattern Recognition, 2002.
[28] P.J. Phillips, A. Martin, C.L. Wilson, and M. Przybocky, “An Introduction to Evaluating Biometric Systems,” Computer, vol. 33, no. 2, Feb. 2000.
[29] P.J. Phillips, H. Moon, S.A. Rizvi, and P.J. Rauss, “The FERET Evaluation Methodology for Face-Recognition Algorithms,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090-1104, Oct. 2000.
[30] R. Sanchez-Reillo and C. Sanchez-Avila, “Fingerprint Verification Using Smart Cards for Access Control Systems,” IEEE Aerospace and Electronic Systems, vol. 17, no. 9, pp. 12-15, Sept. 2002.
[31] “Best Practices in Testing and Reporting Performance of Biometric Devices,” U.K. Government's Biometrics Working Group, v2.01, Aug. 2002.

Index Terms:
Index Terms- Biometric systems, fingerprint verification, performance evaluation, technology evaluation, FVC.
Citation:
Raffaele Cappelli, Dario Maio, Davide Maltoni, James L. Wayman, Anil K. Jain, "Performance Evaluation of Fingerprint Verification Systems," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 3-18, Jan. 2006, doi:10.1109/TPAMI.2006.20
Usage of this product signifies your acceptance of the Terms of Use.