This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Learning to Transform Time Series with a Few Examples
October 2007 (vol. 29 no. 10)
pp. 1759-1775
ASCII Text
x 
 
Ali Rahimi, Ben Recht, Trevor Darrell, "Learning to Transform Time Series with a Few Examples," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1759-1775, October, 2007.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2007.1001,
author = {Ali Rahimi and Ben Recht and Trevor Darrell},
title = {Learning to Transform Time Series with a Few Examples},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {29},
number = {10},
issn = {0162-8828},
year = {2007},
pages = {1759-1775},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2007.1001},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Learning to Transform Time Series with a Few Examples
IS - 10
SN - 0162-8828
SP1759
EP1775
EPD - 1759-1775
A1 - Ali Rahimi,
A1 - Ben Recht,
A1 - Trevor Darrell,
PY - 2007
KW - Semi-supervised learning
KW - example-based tracking
KW - manifold learning
KW - nonlinear system identification
VL - 29
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
We describe a semi-supervised regression algorithm that learns to transform one time series into another time series given examples of the transformation. This algorithm is applied to tracking, where a time series of observations from sensors is transformed to a time series describing the pose of a target. Instead of defining and implementing such transformations for each tracking task separately, our algorithm learns a memoryless transformation of time series from a few example input-output mappings. The algorithm searches for a smooth function that fits the training examples and, when applied to the input time series, produces a time series that evolves according to assumed dynamics. The learning procedure is fast and lends itself to a closed-form solution. It is closely related to nonlinear system identification and manifold learning techniques. We demonstrate our algorithm on the tasks of tracking RFID tags from signal strength measurements, recovering the pose of rigid objects, deformable bodies, and articulated bodies from video sequences. For these tasks, this algorithm requires significantly fewer examples compared to fully-supervised regression algorithms or semi-supervised learning algorithms that do not take the dynamics of the output time series into account.

[1] J.B. Tenenbaum, V. de Silva, and J.C. Langford, “A Global Geometric Framework for Nonlinear Dimensionality Reduction,” Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
[2] S. Roweis and L. Saul, “Nonlinear Dimensionality Reduction by Locally Linear Embedding,” Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
[3] M. Belkin and P. Niyogi, “Laplacian Eigenmaps for Dimensionality Reduction and Data Representation,” Neural Computation, vol. 15, no. 6, pp. 1373-1396, 2003.
[4] D. Donoho and C. Grimes, “Hessian Eigenmaps: New Locally Linear Embedding Techniques for Highdimensional Data,” Technical Report TR2003-08, Dept. of Statistics, Stanford Univ., 2003.
[5] K. Weinberger and L. Saul, “Unsupervised Learning of Image Manifolds by Semidefinite Programming,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[6] M. Brand, “Charting a Manifold,” Proc. Conf. Neural Information Processing Systems, 2002.
[7] M. Balasubramanian, E.L. Schwartz, J.B. Tenenbaum, V. de Silva, and J.C. Langford, “The Isomap Algorithm and Topological Stability,” Science, vol. 295, no. 5552, 2002.
[8] O. Jenkins and M. Mataric, “A Spatio-Temporal Extension to Isomap Nonlinear Dimension Reduction,” Proc. Int'l Conf. Machine Learning, 2004.
[9] J. Ham, D. Lee, and L. Saul, “Learning High Dimensional Correspondences from Low Dimensional Manifolds,” Proc. Int'l Conf. Machine Learning, 2003.
[10] R. Pless and I. Simon, “Using Thousands of Images of an Object,” Proc. Conf. Computer Vision, Pattern Recognition, and Image Processing, 2002.
[11] X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions,” Proc. Int'l Conf. Machine Learning, 2003.
[12] M. Belkin, I. Matveeva, and P. Niyogi, “Regularization and Semi-Supervised Learning on Large Graphs,” Proc. 17th Ann. Conf. Computational Learning Theory, 2004.
[13] J. Lafferty, A. McCallum, and F. Pereira, “Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data,” Proc. Int'l Conf. Machine Learning, pp. 282-289, 2001.
[14] A. Smola, S. Mika, B. Schoelkopf, and R.C. Williamson, “Regularized Principal Manifolds,” J. Machine Learning, vol. 1, pp. 179-209, 2001.
[15] A. Rahimi, B. Recht, and T. Darrell, “Learning Appearance Manifolds from Video,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[16] Z. Ghahramani and S. Roweis, “Learning Nonlinear Dynamical Systems Using an EM Algorithm,” Neural Information Processing Systems (NIPS), pp. 431-437, 1998.
[17] H. Valpola and J. Karhunen, “An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models,” Neural Computation, vol. 14, no. 11, pp. 2647-2692, 2002.
[18] L. Ljung, System Identification: Theory for the User. Prentice-Hall, 1987.
[19] A. Juditsky, H. Hjalmarsson, A. Benveniste, B. Delyon, L. Ljung, J. Sjöberg, and Q. Zhang, “Nonlinear Black-Box Models in System Identification: Mathematical Foundations,” Automatica, vol. 31, no. 12, pp. 1725-1750, 1995.
[20] K.-C. Lee and D. Kriegman, “Online Learning of Probabilistic Appearance Manifolds for Video-Based Recognition and Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[21] G. Doretto, A. Chiuso, and Y.W.S. Soatto, “Dynamic Textures,” Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, 2003.
[22] O. Bousquet and A. Elisseeff, “Stability and Generalization,” Journal of Machine Learning Research, 2002.
[23] M.P.T. Evgeniou and T. Poggio, “Regularization Networks and Support Vector Machines,” Advances in Computational Math., 2000.
[24] G. Wahba, “Spline Models for Observational Data,” Proc. SIAM, vol. 59, 1990.
[25] B. Schölkopf, R. Herbrich, A. Smola, and R. Williamson, “A Generalized Representer Theorem,” Proc. Neural Networks and Computational Learning Theory, NeuroCOLT Technical Report NC-TR-00-081, 2000.
[26] V. Vapnik, Statistical Learning Theory. Wiley, 1998.
[27] T. De Bie and N. Cristianini, “Convex Methods for Transduction,” Proc. Conf. Neural Information Processing Systems, 2003.
[28] T. Joachims, “Transductive Inference for Text Classification Using Support Vector Machines,” Proc. Int'l Conf. Machine Learning, pp.200-209, 1999.
[29] K. Bennett and A. Demiriz, “Semi-Supervised Support Vector Machines,” Proc. Conf. Advances in Neural Information Processing Systems, pp. 368-374, 1998.
[30] R. Rifkin, G. Yeo, and T. Poggio, “Regularized Least Squares Classification,” Proc. Conf. Advances in Learning Theory: Methods, Model and Applications, NATO Science Series III: Computer and Systems Sciences, vol. 190, 2003.
[31] T. Kailath, A.H. Sayed, and B. Hassibi, Linear Estimation. Prentice Hall, 2000.
[32] A. Rahimi, “Learning to Transform Time Series with a Few Examples,” PhD dissertation, Massachusetts Inst. of Technology, Computer Science and AI Lab, Cambridge, 2005.
[33] D. Fokkema, G. Sleijpen, and H. van der Vorst, “Jacobi-Davidson Style Qr and Qz Algorithms for the Reduction of Matrix Pencils,” SIAM J. Scientific Computing, vol. 20, no. 1, pp. 94-125, 1998.
[34] J. Patten, B. Recht, and H. Ishii, “Audiopad: A Tag-Based Interface for Musical Performance,” Proc. Conf. New Interfaces for Musical Expression, 2002.
[35] J. Patten, H. Ishii, J. Hines, and G. Pangaro, “Sensetable: A Wireless Object Tracking Platform for Tangible User Interfaces,” Proc. Conf. Human Factors in Computing Systems, 2001.
[36] H. Sidenbladh, M.J. Black, and D. Fleet, “Stochastic Tracking of 3D Human Figures Using 2D Image Motion,” Proc. European Conf. Computer Vision, pp. 702-718, 2000.
[37] C. Bregler and J. Malik, “Tracking People with Twists and Exponential Maps,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1998.
[38] A. Rahimi, “Spline Drawing Tool for MATLAB,” MIT CSAIL http://people.csail.mit.edu/rahimi/splines html, technical report, Aug. 2005.
[39] A.A. Efros, A.C. Berg, G. Mori, and J. Malik, “Recognizing Action at a Distance,” Proc. Int'l Conf. Computer Vision, 2003.
[40] G. Shakhnarovich, P. Viola, and T. Darrell, “Fast Pose Estimation with Parameter Sensitive Hashing,” Proc. Int'l Conf. Computer Vision, 2003.
[41] A.M. Elgammal, “Learning to Track: Conceptual Manifold Map for Closed-Form Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 724-730, 2005.
[42] A. Agarwal and B. Triggs, “3D Human Pose from Silhouettes by Relevance Vector Regression,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[43] A. Agarwal, A. Hertzmann, and S.M.S.D.H. Salesin, “Keyframe-Based Tracking for Rotoscoping and Animation,” Proc. ACM SIGGRAPH, 2004.
[44] A. Agarwala, “Snaketoonz: A Semi-Automatic Approach to Creating Cel Animation from Video,” Proc. Int'l Symp. Non-Photorealistic Animation and Rendering, http://www.agarwala.org/Pagessnaketoonz.html , 2002.
[45] K. Grauman and T. Darrell, “Fast Contour Matching Using Approximate Earth Mover's Distance,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[46] A. Rahimi and B. Recht, “Estimating Observation Functions in Dynamical Systems Using Unsupervised Regression,” Proc. Advances in Neural Information Processing Systems, 2006.
[47] K. Grauman and T. Darrell, “The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features,” Proc. Int'l Conf. Computer Vision, 2005.
[48] N. Roy and A. McCallum, “Toward Optimal Active Learning through Sampling Estimation of Error Reduction,” Proc. IEEE Int'l Conf. Machine Learning, pp. 441-448, 2001.
[49] G. Schohn and D. Cohn, “Less Is More: Active Learning with Support Vector Machines,” Proc. IEEE Int'l Conf. Machine Learning, pp. 839-846, 2000.
[50] R. Yan, J. Yang, and A. Hauptmann, “Automatically Labeling Video Data Using Multi-Class Active Learning,” Proc. Int'l Conf. Computer Vision, pp. 516-523, 2003.
[51] D.P. Bertsekas, A. Nedic, and A.E. Ozdaglar, Convex Analysis and Optimization. Athena Scientific, 2001.

Index Terms:
Semi-supervised learning, example-based tracking, manifold learning, nonlinear system identification
Citation:
Ali Rahimi, Ben Recht, Trevor Darrell, "Learning to Transform Time Series with a Few Examples," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1759-1775, Oct. 2007, doi:10.1109/TPAMI.2007.1001
Usage of this product signifies your acceptance of the Terms of Use.