This Article 
 Bibliographic References 
 Add to: 
A Curve Fitting Problem and Its Application in Modeling Objects in Monocular Image Sequences
May 2002 (vol. 24 no. 5)
pp. 674-686

In this paper, we present a solution to a particular curve (surface) fitting problem and demonstrate its application in modeling objects from monocular image sequences. The curve fitting algorithm is based on a modified nonparametric regression method, which forms the core contribution of this work. This method is far more effective compared to standard estimation techniques, such as the maximum likelihood estimation method, and can take into account the discontinuities present in the curve. Next, the theoretical results of this 1D curve estimation technique are extended significantly for an object modeling problem. Here, the input to the algorithm is a monocular image sequence of an object undergoing rigid motion. By using the affine camera projection geometry and a given choice of an image frame pair in the sequence, we adopt the KvD model to express the depth at each point on the object as a function of the unknown out-of-plane rotation, and some measurable quantities computed directly from the optical flow. This is repeated for multiple image pairs (keeping one fixed image frame which we formally call the base image and choosing another frame from the sequence). The depth map is next estimated from these equations using the modified nonparametric regression analysis. We conducted experiments on various image sequences to verify the effectiveness of the technique. The results obtained using our curve fitting technique can be refined further by hierarchical techniques, as well as by nonlinear optimization techniques in structure from motion.

[1] K. Aizawa and T.S. Huang, “Model-Based Image Coding: Advanced Video Coding Techniques for Very Low Bit-Rate Application,” Proc. IEEE, vol. 83, pp. 259-271, Aug. 1995.
[2] T. Akimoto, Y. Suenaga, and R.S. Wallace, “Automatic Creation of 3-D Facial Models,” IEEE Trans. Computer Graphics and Applications, vol. 13, pp. 16-22, Sept. 1993.
[3] M. Black and P. Anandan, The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields J. Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
[4] G. Bozdagi, A.M. Tekalp, and L. Onural, “Simultaneous 3-D Motion Estimation and Wire-Frame Model Adaptation for Knowledge-Based Video Coding,” IEEE Trans. Circuits and Systems for Video Technology, vol. 4, pp. 413-416, 1994.
[5] T.E. Boult and J.R. Kender, “Visual Surface Reconstruction Using Sparse Depth Data, Proceedings,” Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition, pp. 68-76, 1986.
[6] P. Burman and R.H. Shumway, “A Semiparametric Approach to Seasonal Time Series Models,” J. Time Series Analysis, vol. 19, pp. 127-145, 1998.
[7] D.R. Forsey and R.H. Bartels, “Surface Fitting with Hierarchical Splines,” Computer Graphics, Apr. 1995.
[8] P. Fua and C. Miccio, “Animated Heads from Ordinary Images: A Least-Squares Approach,” Computer Vision and Image Understanding, vol. 75, no. 3, pp. 247-259, Sept. 1999.
[9] X. Hu and N. Ahuja, "Motion Estimation Under Orthographic Projection," IEEE Trans. Robotics and Automation, vol. 7, no. 6, pp. 848-853, June 1991.
[10] T.S. Huang and C.H. Lee, "Motion and Structure From Orthographic Projections," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, pp. 536-540, 1989.
[11] C.K. Heng and K. Sengupta, “Bootstraping the Tomasi Kanade Algorithm,” Technical Report NUS-KS-0101, Dept. of Electrical Eng., Nat'l Univ. of Singapore (downloadable
[12] J.J. Koenderink and A.J. van Doorn, “Affine Structure from Motion,” J. Optical Soc. Am., vol. 8, no. 2, pp. 377-385, 1991.
[13] J.S. Simonoff, Smoothing Methods in Statistics. Springer-Verlag, 1996.
[14] S.S. Sinha and B.G. Schunck, "Surface Approximation Using Weighted Splines," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 44-49,Lahaina, Hawaii., June 1991.
[15] S. Ullman and R. Basri, "Recognition by Linear Combinations of Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, pp. 992-1006, 1991.
[16] K. Sengupta, Q. Ying, and C.C. Ko, “Human Face Reconstruction Using Heirarchical B-Splines,” Proc. IEEE Int'l Conf. Multimedia and Expo., pp. 361-364, 2001.
[17] L.S. Shaprio, A. Zisserman, and M. Brady, "3D Motion Recovery Via Affine Epipolar Geometry," Int'l J. Computer Vision, vol. 16, pp. 147-182, 1995.
[18] T. Jebara and A. Pentland, “Parameterized Structure from Motion for 3D Adaptive Feedback Tracking of Faces,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 144-150, June 1997.
[19] C. Tomasi and T. Kanade, "Shape and Motion From Image Streams Under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[20] N.M. Vaidya and K.L. Boyer, “Discontinuity-Preserving Surface Reconstruction Using Stochastic Differential Equations,” Computer Vision and Image Understanding, vol. 72, no. 3, pp. 257-270, 1998.
[21] S. Ullman, The Interpretation of Visual Motion. MIT Press, 1979.
[22] K. Sengupta, W. Shiqin, C.C. Ko, and P. Burman, “Automatic Face Modeling from Monocular Image Sequences Using Modified Nonparametric Regression and an Affine Camera Model,” Proc. Fourth IEEE Int'l Conf. Face and Gesture Recognition, pp. 424-529, 2000.
[23] J.Y. Zheng, “Acquiring 3D Models from Sequences of Contours,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 2, pp. 163-178, Feb. 1994.

Index Terms:
Curve fitting, splines, regression, face modeling
K. Sengupta, P. Burman, "A Curve Fitting Problem and Its Application in Modeling Objects in Monocular Image Sequences," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 674-686, May 2002, doi:10.1109/34.1000240
Usage of this product signifies your acceptance of the Terms of Use.