CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2013 vol.35 Issue No.04 - April

Subscribe

Issue No.04 - April (2013 vol.35)

pp: 911-924

M. Ranjbar , Sch. of Comput. Sci., Simon Fraser Univ., Burnaby, BC, Canada

Tian Lan , Sch. of Comput. Sci., Simon Fraser Univ., Burnaby, BC, Canada

Yang Wang , Dept. of Comput. Sci., Univ. of Manitoba, Winnipeg, MB, Canada

S. N. Robinovitch , Sch. of Eng. Sci., Simon Fraser Univ., Burnaby, BC, Canada

Ze-Nian Li , Sch. of Comput. Sci., Simon Fraser Univ., Burnaby, BC, Canada

G. Mori , Sch. of Comput. Sci., Simon Fraser Univ., Burnaby, BC, Canada

ABSTRACT

We develop an algorithm for structured prediction with nondecomposable performance measures. The algorithm learns parameters of Markov Random Fields (MRFs) and can be applied to multivariate performance measures. Examples include performance measures such as $(F_{\beta })$ score (natural language processing), intersection over union (object category segmentation), Precision/Recall at k (search engines), and ROC area (binary classifiers). We attack this optimization problem by approximating the loss function with a piecewise linear function. The loss augmented inference forms a Quadratic Program (QP), which we solve using LP relaxation. We apply this approach to two tasks: object class-specific segmentation and human action retrieval from videos. We show significant improvement over baseline approaches that either use simple loss functions or simple scoring functions on the PASCAL VOC and H3D Segmentation datasets, and a nursing home action recognition dataset.

INDEX TERMS

Loss measurement, Piecewise linear approximation, Labeling, Training, Vectors, Prediction algorithms, Optimization,structural SVM, Optimization, large-margin

CITATION

M. Ranjbar, Tian Lan, Yang Wang, S. N. Robinovitch, Ze-Nian Li, G. Mori, "Optimizing Nondecomposable Loss Functions in Structured Prediction",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.35, no. 4, pp. 911-924, April 2013, doi:10.1109/TPAMI.2012.168REFERENCES

- [1] D. Hoiem, A.A. Efros, and M. Hebert, "Closing the Loop in Scene Interpretation,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.- [2] M.B. Blaschko and C.H. Lampert, "Learning to Localize Objects with Structured Output Regression,"
Proc. 10th European Conf. Computer Vision, 2008.- [3] C. Desai, D. Ramanan, and C. Fowlkes, "Discriminative Models for Multi-Class Object Layout,"
Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.- [4] J. Malik, S. Belongie, T. Leung, and J. Shi, "Contour and Texture Analysis for Image Segmentation,"
Int'l J. Computer Vision, vol. 43, pp. 7-27, 2001.- [5] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts,"
IEEE Trans Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.- [6] M. Szummer, P. Kohli, and D. Hoiem, "Learning CRFs Using Graph Cuts,"
Proc. 10th European Conf. Computer Vision, 2008.- [7] M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, and A. Zisserman, "The Pascal Visual Object Classes (VOC) Challenge,"
Int'l J. Computer Vision, vol. 88, no. 2, pp. 303-338, June 2010.- [8] M. Ranjbar, G. Mori, and Y. Wang, "Optimizing Complex Loss Functions in Structured Prediction,"
Proc. 11th European Conf. Computer Vision, 2010.- [9] B. Taskar, C. Guestrin, and D. Koller, "Max-Margin Markov Networks,"
Proc. Neural. Information Processing Systems Conf., 2003.- [10] I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun, "Large Margin Methods for Structured and Interdependent Output Variables,"
J. Machine Learning Research, vol. 6, pp. 1453-1484, Sept. 2005.- [11] C.H. Teo, A. Smola, S.V. Vishwanathan, and Q.V. Le, "A Scalable Modular Convex Solver for Regularized Risk Minimization,"
Proc. 13th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, 2007.- [12] B. Taskar, S. Lacoste-julien, and M.I. Jordan, "Structured Prediction via the Extragradient Method,"
Proc. Neural. Information Processing Systems Conf., 2005.- [13] S. Shalev-Shwartz, Y. Singer, and N. Srebro, "Pegasos: Primal Estimated Sub-Gradient Solver for SVM,"
Proc. 24th Int'l Conf. Machine Learning, 2007.- [14] B. Taskar, V. Chatalbashev, D. Koller, and C. Guestrin, "Learning Structured Prediction Models: A Large Margin Approach,"
Proc. 22nd Int'l Conf. Machine Learning, 2005.- [15] T. Joachims, "A Support Vector Method for Multivariate Performance Measures,"
Proc. 22nd Int'l Conf. Machine Learning, 2005.- [16] Y. Yue, T. Finley, F. Radlinski, and T. Joachims, "A Support Vector Method for Optimizing Average Precision,"
Proc. 30th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, 2007.- [17] S. Chakrabarti, R. Khanna, U. Sawant, and C. Bhattacharyya, "Structured Learning for Non-Smooth Ranking Losses,"
Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, 2008.- [18] D. McAllester, T. Hazan, and J. Keshet, "Direct Loss Minimization for Structured Prediction,"
Proc. Neural. Information Processing Systems Conf., 2010.- [19] T. Finley and T. Joachims, "Training Structural SVMs When Exact Inference Is Intractable,"
Proc. 25th Int'l Conf. Machine Learning, 2008.- [20] I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun, "Support Vector Machine Learning for Interdependent and Structured Output Spaces,"
Proc. 21st Int'l Conf. Machine Learning, 2004.- [21] T. Werner, "A Linear Programming Approach to Max-Sum Problem: A Review,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1165-1179, July 2007.- [22] N. Komodakis, N. Paragios, and G. Tziritas, "MRF Energy Minimization and Beyond via Dual Decomposition,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 3, pp. 531-552, Mar. 2011.- [23] Y. Boykov and V. Kolmogorov, "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no 9, pp. 1124-1137, Sept. 2004.- [24] T. Do and T. Artieres, "Large Margin Training for Hidden Markov Models with Partially Observed States,"
Proc. 26th Int'l Conf. Machine Learning, 2009.- [25] O. Meshi, D. Sontag, T. Jaakkola, and A. Globerson, "Learning Efficiently with Approximate Inference via Dual Losses,"
Proc. Int'l Conf. Machine Learning, 2010.- [26] MeshLab, http:/meshlab.sourceforge.net/, 2012.
- [27] M. Garland and P.S. Heckbert, "Surface Simplification Using Quadric Error Metrics,"
Proc. ACM Siggraph, 1997.- [28] P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Graph-Based Image Segmentation,"
Int'l J. Computer Vision, vol. 59, no. 2, pp. 167-181, 2004.- [29] K.E.A. van de Sande, T. Gevers, and C.G.M. Snoek, "Evaluating Color Descriptors for Object and Scene Recognition,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1582-1596, Sept. 2010.- [30] B. Leibe, A. Leonardis, and B. Schiele, "Combined Object Categorization and Segmentation with an Implicit Shape Model,"
Proc. ECCV Workshop Statistical Learning in Computer Vision, 2004.- [31] P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan, "Object Detection with Discriminatively Trained Part Based Models,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, Sept. 2009.- [32] M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, and A. Zisserman, "The PASCAL Visual Object Classes Challenge 2009 (VOC2009) Results," http://www.pascal-network.org/ challenges/ VOC/voc2009/workshopindex.html, 2009.
- [33] M. Everingham, L. Van Gool, C.K. Williams, J. Winn, and A. Zisserman, "The PASCAL Visual Object Classes Challenge 2010 (VOC2010) Results," http://www.pascal-network.org/ challenges/ VOC/voc2010/workshopindex.html, 2010.
- [34] L. Bourdev and J. Malik, "Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations,"
Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.- [35] C. Stauffer and W.E.L. Grimson, "Learning Patterns of Activity Using Real-Time Tracking,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 747-757, Aug. 2000.- [36] N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection,"
Proc. IEEE Computer Vision and Pattern Recognition Conf., 2005.- [37] C.C. Loy, T. Xiang, and S. Gong, "Modelling Activity Global Temporal Dependencies Using Time Delayed Probabilistic Graphical Model,"
Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.- [38] T. Lan, Y. Wang, G. Mori, and S. Robinovitch, "Retrieving Actions in Group Contexts,"
Proc. Int'l Workshop Sign Gesture Activity, 2010.- [39] J. Demsar, "Statistical Comparisons of Classifiers over Multiple Data Sets,"
J. Machine Learning Research, vol. 7, pp. 1-30, 2006. |