This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
On the Use of Error Propagation for Statistical Validation of Computer Vision Software
October 2005 (vol. 27 no. 10)
pp. 1603-1614
ASCII Text
x 
 
Xufei Liu, Tapas Kanungo, Robert M. Haralick, "On the Use of Error Propagation for Statistical Validation of Computer Vision Software," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1603-1614, October, 2005.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2005.203,
author = {Xufei Liu and Tapas Kanungo and Robert M. Haralick},
title = {On the Use of Error Propagation for Statistical Validation of Computer Vision Software},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {27},
number = {10},
issn = {0162-8828},
year = {2005},
pages = {1603-1614},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2005.203},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - On the Use of Error Propagation for Statistical Validation of Computer Vision Software
IS - 10
SN - 0162-8828
SP1603
EP1614
EPD - 1603-1614
A1 - Xufei Liu,
A1 - Tapas Kanungo,
A1 - Robert M. Haralick,
PY - 2005
KW - Index Terms- Statistical analysis
KW - multivariate hypothesis testing
KW - 3D parameter estimation
KW - error propagation
KW - software validation
KW - software engineering.
VL - 27
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Computer vision software is complex, involving many tens of thousands of lines of code. Coding mistakes are not uncommon. When the vision algorithms are run on controlled data which meet all the algorithm assumptions, the results are often statistically predictable. This renders it possible to statistically validate the computer vision software and its associated theoretical derivations. In this paper, we review the general theory for some relevant kinds of statistical testing and then illustrate this experimental methodology to validate our building parameter estimation software. This software estimates the 3D positions of buildings vertices based on the input data obtained from multi-image photogrammetric resection calculations and 3D geometric information relating some of the points, lines and planes of the buildings to each other.

[1] M. Altman and M. McDonald, “Choosing Reliable Statistical Software,” PS: Political Science and Politics, vol. 24, pp. 681-688, 2002, http://www.hmdc.harvard.edu/micah_altman/ numaltpm3.pdf.
[2] T.W. Anderson, Introduction to Multivariate Statistical Analysis. New York: John Wiley & Sons, 1984.
[3] S.F. Arnold, Mathematical Statistics. Prentice-Hall, 1990.
[4] D.P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods. New York: Academic Press, 1982.
[5] P.A. Bromiley, M. Pokrik, and N.A. Thacker, “Computing Covariances for Mutual Information Co-Registration,” Proc. Medical Image Analysis Conf., 2004.
[6] G. Casella and R.L. Berger, Statistical Inference. Wadsworth, 1990.
[7] A. Chen, R. Drachenberg, and J. Symanzik, “Assessing the Reliability of Web-Based Statistical Software,” Computational Statistics, vol. 18, pp. 107-122, 2003, http://www.math.usu. edu/~symanzik/papers 2003_computstat.pdf.
[8] K. Cho, P. Meer, and J. Cabrera, “Performance Assessment through Bootstrap,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp. 1185-1198, 1997.
[9] A.R. Chowdhury and R. Chellappa, “Statistical Error Propagation in 3D Modeling from Monocular Video,” Proc. IEEE Workshop Statistical Analysis in Computer Vision, 2003, http://www.cse. lehigh. edu/~rjm2/SACV/papers chowdhury-chellappa.pdf.
[10] P. Courtney and N.A. Thacker, “Performance Characterisation in Computer Vision: Statistics in Testing and Design,” Imaging and Vision Systems: Theory, Assessment and Applications, pp. 109-128, 2001, http://peipa.essex.ac.uk/benchmark/tutorials/ epsrcss2002/handoutepsrcs%s2002-handout.pdf .
[11] S. Crossley, N.A. Thacker, and N.L. Seed, “Benchmarking Physical Scene Modelling,” Proc. British Machine Vision Conf. (BMVC), pp. 346-355, 1998.
[12] B. Efron and R. Tibshirani, An Introduction to the Bootstrap. Chapman and Hall, 1993.
[13] O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint. Cambridge, Mass.: MIT Press, 1993.
[14] R. Fletcher, Practical Methods of Optimization. New York: John Wiley & Sons, 1987.
[15] J. Ghezzi, M. Jazayeri, and D. Mandrioli, Fundamentals of Software Engineering. Pearson Education, 2002.
[16] P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization. New York: Academic Press, 1981.
[17] R.M. Haralick, “Propagating Covariance in Computer Vision,” Proc. IAPR Int'l Conf. Pattern Recognition, pp. 493-498, Oct. 1994.
[18] D.A. Hudson, “Inverse Perspective Projection Algorithm: Error Estimation,” master's thesis, Univ. of Washington, Seattle, 1990.
[19] K. Kanatani, Statistical Optimization for Geometric Computation: Theory and Practice. Elsivier Science, 1996.
[20] T. Kanungo and R.M. Haralick, “Multivariate Hypothesis Testing for Gaussian Data: Theory and Software,” Technical Report ISL: ISL-TR-95-05, Univ. of Washington, Seattle, Oct. 1995. Software available from: http://lib.stat.cmu.edu/generalmvntest.gz .
[21] T. Kanungo, M.Y. Jaisimha, J. Palmer, and R.M. Haralick, “A Methodology for Quantitative Performance Evaluation of Detection Algorithms,” IEEE Trans. Image Processing, pp. 1667-1674, 1995.
[22] K. Koch, Parameter Estimation and Hypothesis Testing in Linear Models. New York: Springer-Verlag, 1987.
[23] X. Liu, “Modeling and Performance Characterization of 3D Parameter Estimation Using Perspective Geometry,” PhD thesis, Univ. of Washington, Seattle, 1995.
[24] X. Liu, R.M. Haralick, and A. Bedekar, “Optimization Methods for Estimating 3D Object Parameters,” Proc. 1994 DARPA Image Understanding Workshop, pp. 1017-1025, 1994.
[25] X. Liu, R.M. Haralick, and K. Thornton, “Site Model Construction Using Geometric Constrained Optimization,” Proc. 1996 DARPA Image Understanding Workshop, 1996.
[26] X. Liu, T. Kanungo, and R.M. Haralick, “Statistical Validation of Computer Vision Software,” Proc. 1996 DARPA Image Understanding Workshop, pp. 1533-1540, 1996.
[27] B.D. McCullough, “Assessing the Reliability of Statistical Software: Part I,” The Am. Statistician, pp. 358-366, 1998, http://www.amstat.org/publications/tasmccull-1.pdf .
[28] B.D. McCullough, “Assesing the Reliability of Statistical Software: Part II,” The Am. Statistician, pp. 149-159, 1999, http://www. amstat.org/publications/tasmccull.pdf .
[29] W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling, Numerical Recipes in C. New York: Cambridge Univ. Press, 1990.
[30] L. Wilkinson, “Practical Guidelines for Testing Statistical Software,” Computational Statistics: Papers Collected on the Occasion of the 25th Conf. Statistical Computing at Schloss Reinsensburg, Physica-Verlag, 1994, http://www.spss.com/research/wilkinson/Publications Accuracy.pdf.

Index Terms:
Index Terms- Statistical analysis, multivariate hypothesis testing, 3D parameter estimation, error propagation, software validation, software engineering.
Citation:
Xufei Liu, Tapas Kanungo, Robert M. Haralick, "On the Use of Error Propagation for Statistical Validation of Computer Vision Software," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1603-1614, Oct. 2005, doi:10.1109/TPAMI.2005.203
Usage of this product signifies your acceptance of the Terms of Use.