IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) is a scholarly archival journal published monthly. This journal covers traditional areas of computer vision and image understanding, all traditional areas of pattern analysis and recognition, and selected areas of machine intelligence. Read the full scope of TPAMI.


Expand your horizons with Colloquium, a monthly survey of abstracts from all CS transactions! Replaces OnlinePlus in January 2017.


From the September 2018 issue

Learning from Narrated Instruction Videos

By Jean-Baptiste Alayrac, Piotr Bojanowski, Nishant Agrawal, Josef Sivic, Ivan Laptev, and Simon Lacoste-Julien

Featured article thumbnail imageAutomatic assistants could guide a person or a robot in performing new tasks, such as changing a car tire or repotting a plant. Creating such assistants, however, is non-trivial and requires understanding of visual and verbal content of a video. Towards this goal, we here address the problem of automatically learning the main steps of a task from a set of narrated instruction videos. We develop a new unsupervised learning approach that takes advantage of the complementary nature of the input video and the associated narration. The method sequentially clusters textual and visual representations of a task, where the two clustering problems are linked by joint constraints to obtain a single coherent sequence of steps in both modalities. To evaluate our method, we collect and annotate a new challenging dataset of real-world instruction videos from the Internet. The dataset contains videos for five different tasks with complex interactions between people and objects, captured in a variety of indoor and outdoor settings. We experimentally demonstrate that the proposed method can automatically discover, learn and localize the main steps of a task in input videos.

download PDF View the PDF of this article      csdl View this issue in the digital library


Editorials and Announcements

Announcements

  • TPAMI now offers authors access to Code Ocean. Code Ocean is a cloud-based executable research platform that allows authors to share their algorithms in an effort to make the world’s scientific code more open and reproducible. Learn more or sign up for free.
  • We are pleased to announce that Sven Dickinson, a professor in the Department of Computer Science at the University of Toronto, Canada, has been named the new Editor-in-Chief of the IEEE Transactions on Pattern Analysis and Machine Intelligence starting in 2017.
  • According to Clarivate Analytics' 2016 Journal Citation Report, TPAMI has an impact factor of 8.329.

Editorials


Guest Editorials


Call for Papers


Reviewers List


Annual Index


Access recently published TPAMI articles

RSS Subscribe to the RSS feed of recently published TPAMI content

mail icon Sign up for e-mail notifications through IEEE Xplore Content Alerts

preprints icon View TPAMI preprints in the Computer Society Digital Library