Issue No. 04 - April (2012 vol. 34)
F. Bach , Lab. d'Inf., Ecole Normale Supe'rieure, Paris, France
J. Mairal , Dept. of Stat., Univ. of California, Berkeley, CA, USA
J. Ponce , INRIA-Willow Project-Team, Ecole Normale Super., Paris, France
Modeling data with linear combinations of a few elements from a learned dictionary has been the focus of much recent research in machine learning, neuroscience, and signal processing. For signals such as natural images that admit such sparse representations, it is now well established that these models are well suited to restoration tasks. In this context, learning the dictionary amounts to solving a large-scale matrix factorization problem, which can be done efficiently with classical optimization tools. The same approach has also been used for learning features from data for other purposes, e.g., image classification, but tuning the dictionary in a supervised way for these tasks has proven to be more difficult. In this paper, we present a general formulation for supervised dictionary learning adapted to a wide variety of tasks, and present an efficient algorithm for solving the corresponding optimization problem. Experiments on handwritten digit classification, digital art identification, nonlinear inverse image problems, and compressed sensing demonstrate that our approach is effective in large-scale settings, and is well suited to supervised and semi-supervised classification, as well as regression tasks for data that admit sparse representations.
regression analysis, compressed sensing, data models, handwritten character recognition, image classification, image representation, image restoration, learning (artificial intelligence), matrix decomposition, regression tasks, task-driven dictionary learning, data modeling, linear combinations, learned dictionary, machine learning, neuroscience, signal processing, natural images, sparse representations, restoration tasks, large-scale matrix factorization problem, classical optimization tools, image classification, supervised dictionary learning, handwritten digit classification, digital art identification, nonlinear inverse image problems, compressed sensing, semisupervised classification, Dictionaries, Sparse matrices, Vectors, Sensors, Cost function, Machine learning, compressed sensing., Basis pursuit, Lasso, dictionary learning, matrix factorization, semi-supervised learning
F. Bach, J. Mairal and J. Ponce, "Task-Driven Dictionary Learning," in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 34, no. , pp. 791-804, 2012.