The Community for Technology Leaders
Multimodal Interfaces, IEEE International Conference on (2002)
Pittsburgh, Pennsylvania
Oct. 14, 2002 to Oct. 16, 2002
ISBN: 0-7695-1834-6
TABLE OF CONTENTS
Introduction

Preface (PDF)

pp. xi
Perceptual Computing Systems and Methods

Layered Representations for Human Activity Recognition (Abstract)

Nuria Oliver , Microsoft Research
Ashutosh Garg , University of Illinois at Urbana-Champaign
Eric Horvitz , Microsoft Research
pp. 3

Evaluating Integrated Speech- and Image Understanding (Abstract)

S. Wachsmuth , Bielefeld University
G. Sagerer , Bielefeld University
C. Bauckhage , Bielefeld University
K. J. Rohlfing , Bielefeld University
J. Fritsch , Bielefeld University
pp. 9

Techniques for Interactive Audience Participation (Abstract)

Randy Pausch , Carnegie Mellon University
Steve Seitz , University of Washington
Dan Maynes-Aminzade , Massachusetts Institute of Technology
pp. 15

Perceptual Collaboration in Neem (Abstract)

P. Barthelmess , University of Colorado at Boulder
C. A. Ellis , University of Colorado at Boulder
pp. 21

A Tracking Framework for Collaborative Human Computer Interaction (Abstract)

E. Polat , Pennsylvania State University
M. Yeasin , Pennsylvania State University
R. Sharma , Pennsylvania State University
pp. 27

A Structural Approach to Distance Rendering in Personal Auditory Displays (Abstract)

Federico Fontana , University of Verona
Laura Ottaviani , University of Verona
Davide Rocchesso , University of Verona
pp. 33

A Multimodal Electronic Travel Aid Device (Abstract)

A. Panuccio , Università degli Studi di Verona
D. Rocchesso , Università degli Studi di Verona
V. Murino , Università degli Studi di Verona
A. Fusiello , Università degli Studi di Verona
F. Fontana , Università degli Studi di Verona
pp. 39
Speech Generation and Recognition

Lecture and Presentation Tracking in an Intelligent Meeting Room (Abstract)

Ivica Rogina , Universität Karlsruhe
Thomas Schaaf , Universität Karlsruhe
pp. 47

Parallel Computing-Based Architecture for Mixed-Initiative Spoken Dialogue (Abstract)

Koji Iwano , Tokyo Institute of Technology
Ryuta Taguma , Tokyo Institute of Technology
Tatsuhiro Moriyama , Tokyo Institute of Technology
Sadaoki Furui , Tokyo Institute of Technology
pp. 53

3-D N-Best Search for Simultaneous Recognition of Distant-Talking Speech of Multiple Talkers (Abstract)

Panikos Heracleous , ATR Spoken Language Translation Research Laboratories
Satoshi Nakamura , ATR Spoken Language Translation Research Laboratories
pp. 59

Integration of Tone Related Feature for Chinese Speech Recognition (Abstract)

Man-Hung Siu , Hong Kong University of Science and Technology
Pui-Fung Wong , Hong Kong University of Science and Technology
pp. 64

Talking Heads: Which Matching between Faces and Synthetic Voices? (Abstract)

Noël Chateau , France Télécom R&D
Marc Mersiol , France Télécom R&D
Valérie Maffiolo , France Télécom R&D
pp. 69

Robust Noisy Speech Recognition with Adaptive Frequency Bank Selection (Abstract)

Ye Tian , Tsinghua University
Ji Wu , Tsinghua University
Dajin Lu , Tsinghua University
Zuoying Wang , Tsinghua University
pp. 75

Covariance-Tied Clustering Method In Speaker Identification (Abstract)

Peng Ding , Chinese Academy of Science
ZhiQiang Wang , Chinese Academy of Science
Yang Liu , Chinese Academy of Science
Xu Bo , Chinese Academy of Science
pp. 81
Dialogue Processing and Language Understanding

Context-Based Multimodal Input Understanding in Conversational Systems (Abstract)

Michelle X. Zhou , IBM T.J. Watson Research Center
Joyce Chai , IBM T.J. Watson Research Center
Keith Houck , IBM T.J. Watson Research Center
Shimei Pan , IBM T.J. Watson Research Center
pp. 87

Context-Sensitive Help for Multimodal Dialogue (Abstract)

Michael Johnston , AT&T Labs - Research
Patrick Ehlen , AT&T Labs - Research
Helen Wright Hastie , AT&T Labs - Research
pp. 93

Referring to Objects with Spoken and Haptic Modalities (Abstract)

Nadia Bellalem , LORIA Laboratory — France
Laurent Romary , LORIA Laboratory — France
Frédéric Landragin , LORIA Laboratory — France
pp. 99

Towards Visually-Grounded Spoken Language Acquisition (Abstract)

Deb Roy , Massachusetts Institute of Technology
pp. 105

Modeling Output in the EMBASSI Multimodal Dialog System (Abstract)

Christian Elting , European Media Laboratory GmbH
Gregor Möhler , Sony International (Europe) GmbH
pp. 111

Multimodal Dialogue Systems for Interactive TVApplications (Abstract)

Pontus Johansson , Link?ping University
Aseel Ibrahim , Nokia Home Communications and Link?ping University
pp. 117

Viewing and Analyzing Multimodal Human-computer Tutorial Dialogue: A Database Approach (Abstract)

Raghu Chalasani , Carnegie Mellon University
Peng Jia , Carnegie Mellon University
Joseph Beck , Carnegie Mellon University
Jack Mostow , Carnegie Mellon University
Andrew Cuneo , Carnegie Mellon University
pp. 129

Adaptive Dialog Based upon Multimodal Language Acquisition (Abstract)

James Flanagan , Rutgers University
Sorin Dusan , Rutgers University
pp. 135

Integrating Emotional Cues into a Framework for Dialogue Management (Abstract)

Matthias Denecke , Carnegie Mellon University
Hartwig Holzapfel , University of Karlsruhe
Alex Waibel , Carnegie Mellon University
Christian Fuegen , University of Karlsruhe
pp. 141
Signing, Gesturing, and Writing

Gesture Patterns during Speech Repairs (Abstract)

Lei Chen , Purdue University
Mary Harper , Purdue University
Francis Quek , Wright State University
pp. 155

Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures (Abstract)

Mohammed Yeasin , Pennsylvania State University
Rajeev Sharma , Pennsylvania State University
Sanshzar Kettebekov , Pennsylvania State University
pp. 161

Purdue RVL-SLLL ASL Database for Automatic Recognition of American Sign Language (Abstract)

Robin Shay , Purdue University
Ronnie B. Wilbur , Purdue University
Aleix M. Martínez , Ohio State University and Purdue University
Avi C. Kak , Purdue University
pp. 167

Hand Gesture Symmetric Behavior Detection and Analysis in Natural Conversation (Abstract)

David McNeill , University of Chicago
Francis Quek , Wright State University
Yingen Xiong , Wright State University
pp. 179

A Multi-Class Pattern Recognition System for Practical Finger Spelling Translation (Abstract)

Jose L. Hernandez-Rebollar , George Washington University
Robert W. Lindeman , George Washington University
Nicholas Kyriakopoulos , George Washington University
pp. 185

A Map-Based System Using Speech and 3D Gestures for Pervasive Computing (Abstract)

Richard M. Wesson , Oregon Health & Science University
Andrea Corradini , Oregon Health & Science University
Philip R. Cohen , Oregon Health & Science University
pp. 191

Hand Tracking Using Spatial Gesture Modeling and Visual Feedback for a Virtual DJ System (Abstract)

Edward Lin , Carnegie Mellon University
Tsuhan Chen , Carnegie Mellon University
Dan Hook , Carnegie Mellon University
Avinash Baliga , Carnegie Mellon University
Andy Cassidy , Carnegie Mellon University
pp. 197
Translation and Multilingual Interfaces

A PDA-Based Sign Translator (Abstract)

Jing Zhang , Mobile Technologies, LLC
Xilin Chen , Carnegie Mellon University
Alex Waibel , Mobile Technologies, LLC and Carnegie Mellon University
Jie Yang , Carnegie Mellon University
pp. 217

The NESPOLE! Multimodal Interface for Cross-lingual Communication - Experience and Lessons Learned (Abstract)

Loredana Taddei , Aethra Telecomunicazioni
Alon Lavie , Carnegie Mellon University
Erica Costantini , University of Trieste
pp. 223

Research of Machine Learning Method for Specific Information Recognition on the Internet (Abstract)

Tiejun Zhao , Harbin Institute of Technology
Sheng Li , Harbin Institute of Technology
Dequan Zheng , Harbin Institute of Technology
Yi Hu , Harbin Institute of Technology
Hao Yu , Harbin Institute of Technology
pp. 229

Multi-Modal Translation System and Its Evaluation (Abstract)

Satoshi Nakamura , ATR, Spoken Language Translation Research Laboratories.
Shigeo Morishima , Seikei University and ATR, Spoken Language Translation Research Laboratories
pp. 241

Towards Universal Speech Recognition (Abstract)

Zhirong Wang , Carnegie Mellon University
Tanja Schultz , Carnegie Mellon University
Alex Waibel , Carnegie Mellon University
Umut Topkara , Carnegie Mellon University
pp. 247

Improved Named Entity Translation and Bilingual Named Entity Extraction (Abstract)

Fei Huang , Carnegie Mellon University
Stephan Vogel , Carnegie Mellon University
pp. 253
Gaze Tracking and Lipreading

Active Gaze Tracking for Human-Robot Interaction (Abstract)

Alexander Zelinsky , Australian National University
Rowel Atienza , Australian National University
pp. 261

3-D Articulated Pose Tracking for Untethered Diectic Reference (Abstract)

Trevor Darrell , Massachusetts Institute of Technology
David Demirdjian , Massachusetts Institute of Technology
pp. 267

Tracking Focus of Attention in Meetings (Abstract)

Rainer Stiefelhagen , Universität Karlsruhe
pp. 273

A Probabilistic Dynamic Contour Model for Accurate and Robust Lip Tracking (Abstract)

Qiang Wang , Tsinghua University
Guangyou Xu , Tsinghua University
Haizhou Ai , Tsinghua University
pp. 281

Attentional Object Spotting by Integrating Multimodal Input (Abstract)

Dana H. Ballard , University of Rochester
Chen Yu , University of Rochester
Shenghuo Zhu , University of Rochester
pp. 287

Lip Tracking for MPEG-4 Facial Animation (Abstract)

Petar S. Aleksic , Northwestern University
Zhilin Wu , Northwestern University
Aggelos K. Katsaggelos , Northwestern University
pp. 293

Multi-Modal Temporal Asynchronicity Modeling by Product HMMs for Robust (Abstract)

Ken?ichi Kumatani , ATR Spoken Language Translation Research Laboratories and Nara Institute of Science and Technology
Satoshi Tamura , ATR Spoken Language Translation Research Laboratories and Tokyo Institute of Technology
Satoshi Nakamura , ATR Spoken Language Translation Research Laboratories
pp. 305
Applications of Multimodal Interfaces

A Multi-Modal Interface for an Interactive Simulated Vascular Reconstruction System (Abstract)

P. M. A. Sloot , University of Amsterdam
R. G. Belleman , University of Amsterdam
E. V. Zudilova , University of Amsterdam
pp. 313

Universal Interfaces to Multimedia Documents (Abstract)

Ine Langer , Harz University of Applied Studies
Gerhard Weber , University of Kiel
Helen Petrie , City University, London
Keith Gladstone , Royal National Institute for the Blind
Cathy Rundle , Royal National Institute for the Blind
Wendy Fisher , City University, London
Liesbeth Pyfers , Pragma, Hoensbroek
pp. 319

A Video Based Interface to Textual Information for the Visually Impaired (Abstract)

Ramani Duraiswami , University of Maryland at College Park
Antoine Chahine , University of Maryland at College Park
Larry S. Davis , University of Maryland at College Park
Ali Zandifar , University of Maryland at College Park
pp. 325

Modular Approach of Multimodal Integration in a Virtual Environment (Abstract)

George N. Phillips Jr. , University of Wisconsin - Madison
Rajarathinam Arangarasan , University of Wisconsin - Madison
pp. 331

Mobile Multi-Modal Data Services for GPRS Phones and Beyond (Abstract)

Michael Pucher , ftw. Telecommunications Research
Robert Finan , Mobilkom Austria AG & Co KG
Georg Niklfeld , ftw. Telecommunications Research
Wolfgang Eckhart , Sonorys Technology GmbH
pp. 337

Flexi-Modal and Multi-Machine User Interfaces (Abstract)

Jie Zhu , Carnegie Mellon University
Jeffrey Nichols , Carnegie Mellon University
Robert C. Miller , Carnegie Mellon University
Robert Malkin , Carnegie Mellon University
Ben Bostwick , Carnegie Mellon University
Edgar Seemann , Carnegie Mellon University
Brad Myers , Carnegie Mellon University
Choon Hong Peck , Carnegie Mellon University
Alex Waibel , Carnegie Mellon University
Michael Bett , Carnegie Mellon University
Bill Scherlis , Carnegie Mellon University
Dave Kong , Carnegie Mellon University
Matthias Denecke , Carnegie Mellon University
Jie Yang , Carnegie Mellon University
pp. 343

A Real-Time Framework for Natural Multimodal Interaction with Large Screen Displays (Abstract)

R. Sharma , Pennsylvania State University
N. Krahnstoever , Pennsylvania State University
S. Kettebekov , Pennsylvania State University
M. Yeasin , Pennsylvania State University
pp. 349

Embarking on Multimodal Interface Design (Abstract)

Anoop K. Sinha , University of California at Berkeley
James A. Landay , University of California at Berkeley
pp. 355

Multi Modal User Interaction in an Automatic Pool Trainer (Abstract)

Wisdom Kobby Vodzi , Aalborg University
Morten Damm Jensen , Aalborg University
Lars Bo Larsen , Aalborg University
pp. 361

Multimodal Contextual Car-Driver Interface (Abstract)

Asim Smailagic , Carnegie Mellon University
Daniel Siewiorek , Carnegie Mellon University
Matthew Hornyak , Carnegie Mellon University
pp. 367
Sensors, Tools, and Platforms for Multimodal Interfaces

Requirements for Automatically Generating Multi-Modal Interfaces for Complex Appliances (Abstract)

Joseph Hughes , MAYA Design, Inc.
Thomas K. Harris , Carnegie Mellon University
Brad Myers , Carnegie Mellon University
Roni Rosenfeld , Carnegie Mellon University
Michael Higgins , MAYA Design, Inc.
Stefanie Shriver , Carnegie Mellon University
Jeffrey Nichols , Carnegie Mellon University
pp. 377

Articulated Model Based People Tracking Using Motion Models (Abstract)

Liang Wang , Chinese Academy of Sciences
Weiming Hu , Chinese Academy of Sciences
Tieniu Tan , Chinese Academy of Sciences
Huazhong Ning , Chinese Academy of Sciences
pp. 383

Audiovisual Arrays for Untethered Spoken Interfaces (Abstract)

Vibhav Rangarajan , Massachusetts Institute of Technology
Trevor Darrell , Massachusetts Institute of Technology
Kevin Wilson , Massachusetts Institute of Technology
Neal Checka , Massachusetts Institute of Technology
pp. 389

Fingerprint Classification by Directional Fields (Abstract)

Wei Wei Zhang , Chinese Academy of Sciences
Yang Sheng Wang , Chinese Academy of Sciences
Sen Wang , Chinese Academy of Sciences
pp. 395

Towards Vision-Based 3-D People Tracking in a Smart Room (Abstract)

Dirk Focken , Universität Karlsruhe
R. Stiefelhagen , Universität Karlsruhe
pp. 400

Using TouchPad Pressure to Detect Negative Affect (Abstract)

Helena M. Mentis , Cornell University
Geri K. Gay , Cornell University
pp. 406

Musically Expressive Doll in Face-to-Face Communication (Abstract)

Kenji Mase , ATR Media Information Science Laboratories
Tomoko Yonezawa , ATR Media Integration & Communications Research Laboratories
pp. 417

Towards Monitoring Human Activities Using an Omnidirectional Camera (Abstract)

Jie Yang , Carnegie Mellon University
Xilin Chen , Carnegie Mellon University
pp. 423

Smart Platform — A Software Infrastructure for Smart Space (SISS) (Abstract)

Guanyou Xu , Tsinghua University
Yanhua Mao , Tsinghua University
Weikai Xie , Tsinghua University
Yuanchun Shi , Tsinghua University
pp. 429
User Studies and System Evaluation

CATCH-2004 Multi-Modal Browser: Overview Description with Usability Analysis (Abstract)

Ladislav Seredi , IBM Voice Technologies & Systems
Jan Kleindienst , IBM Voice Technologies & Systems
Pekka Kapanen , Nokia Research Center
Janne Bergman , Nokia Research Center
pp. 442

Multimodal Interaction During Multiparty Dialogues: Initial Results (Abstract)

Kelly Krout , Oregon Health & Science University
Rachel Coulston , Oregon Health & Science University
Philip R. Cohen , Oregon Health & Science University
pp. 448

Multi-Modal Embodied Agents Scripting (Abstract)

Abe Mamdani , Imperial College London
Yasmine Arafa , Imperial College London
pp. 454

A Methodology for Evaluating Multimodality in a Home Entertainment System (Abstract)

Stefan Rapp , Sony Advanced Technology Centre
Georg Michelitsch , Sony Advanced Technology Centre
Gregor Moehler , Sony Advanced Technology Centre
Jason Williams , Sony Advanced Technology Centre
pp. 460

Body-Based Interfaces (Abstract)

Gerard J. Kim , Pohang University of Science and Technology
Sung H. Han , Pohang University of Science and Technology
Changseok Cho , Korea Institute of Science and Technology
Huichul Yang , Pohang University of Science and Technology
pp. 466

Evaluation of the Command and Control Cube (Abstract)

Jean-Marie Burkhardt , INRIA - Universite Paris V
Paul Richard , ISTIA - Innovation
Jérôme Grosjean , i3D INRIA
Sabine Coquillart , i3D INRIA
pp. 473

Interruptions as Multimodal Outputs: Which are the Less Disruptive? (Abstract)

Ted Selker , Massachusetts Institute of Technology
Alexandre Stouffs , Massachusetts Institute of Technology
Ernesto Arroyo , Massachusetts Institute of Technology
pp. 479

Experimentally Augmenting an Intelligent Tutoring System with Human-Supplied Capabilities: Adding Human-Provided Emotional Scaffolding to an Automated Reading Tutor that Listens (Abstract)

Barry Kort , Massachusetts Institute of Technology
Rosalind Picard , Massachusetts Institute of Technology
Gregory Aist , Project LISTEN
Jack Mostow , Project LISTEN
Rob Reilly , Massachusetts Institute of Technology
pp. 483

Individual Differences in Facial Expression: Stability over Time, Relation to Self-Reported Emotion, and Ability to Inform Person Identification (Abstract)

Karen Schmidt , University of Pittsburgh
Paul Ekman , University of California at San Francisco
Ralph Gross , Carnegie Mellon University
Jeffrey F. Cohn , University of Pittsburgh and Carnegie Mellon University
pp. 491
Face Detection, Recognition, and Synthesis

Training a Talking Head (Abstract)

Rashid Clark , University of California at Santa Cruz
Dominic W. Massaro , University of California at Santa Cruz
Michael M. Cohen , University of California at Santa Cruz
pp. 499

Labial Coarticulation Modeling for Realistic Facial Animation (Abstract)

Emanuela Magno Caldognetto , Istituto di Scienze e Tecnologie della Cognizione- C.N.R.
Giulio Perin , Universit? di Padova
Piero Cosi , Istituto di Scienze e Tecnologie della Cognizione- C.N.R.
Claudio Zmarich , Istituto di Scienze e Tecnologie della Cognizione- C.N.R.
pp. 505

Improved Information Maximization based Face and Facial Feature Detection from Real-time Video and Application in a Multi-Modal Person Identification System (Abstract)

Ziyou Xiong , University of Illinois at Urbana-Champaign
Thomas S. Huang , University of Illinois at Urbana-Champaign
Roy Wang , University of Illinois at Urbana-Champaign
Yunqiang Chen , University of Illinois at Urbana-Champaign
pp. 511

Animating Arbitrary Topology 3D Facial Model Using the MPEG-4 FaceDefTables (Abstract)

Zhaoqi Wang , Chinese Academy of Sciences
Wen Gao , Chinese Academy of Sciences and Harbin Institute of Technology
Zhiguo Li , Chinese Academy of Sciences
Dalong Jiang , Chinese Academy of Sciences
pp. 517

An Improved Active Shape Model for Face Alignment (Abstract)

Bo Cao , Chinese Academy of Sciences
Wen Gao , Chinese Academy of Sciences and Harbin Institute of Technology
Wei Wang , Beijing Polytechnic University
Shiguang Shan , Chinese Academy of Sciences
Baocai Yin , Beijing Polytechnic University
pp. 523

An Improved Algorithm for Hairstyle Dynamics (Abstract)

Baocai Yin , Beijing Polytechnic University
Dehui Kong , Beijing Polytechnic University
Wenjun Lao , Beijing Polytechnic University
pp. 535
Author Index

Author Index (PDF)

pp. 541
79 ms
(Ver 3.3 (11022016))