The Community for Technology Leaders
Multimodal Interfaces, IEEE International Conference on (2002)
Pittsburgh, Pennsylvania
Oct. 14, 2002 to Oct. 16, 2002
ISBN: 0-7695-1834-6
TABLE OF CONTENTS
Introduction

Preface (PDF)

pp. xi
Perceptual Computing Systems and Methods

Layered Representations for Human Activity Recognition (Abstract)

Nuria Oliver , Microsoft Research
Eric Horvitz , Microsoft Research
Ashutosh Garg , University of Illinois at Urbana-Champaign
pp. 3

Evaluating Integrated Speech- and Image Understanding (Abstract)

C. Bauckhage , Bielefeld University
J. Fritsch , Bielefeld University
K. J. Rohlfing , Bielefeld University
S. Wachsmuth , Bielefeld University
G. Sagerer , Bielefeld University
pp. 9

Techniques for Interactive Audience Participation (Abstract)

Dan Maynes-Aminzade , Massachusetts Institute of Technology
Randy Pausch , Carnegie Mellon University
Steve Seitz , University of Washington
pp. 15

Perceptual Collaboration in Neem (Abstract)

P. Barthelmess , University of Colorado at Boulder
C. A. Ellis , University of Colorado at Boulder
pp. 21

A Tracking Framework for Collaborative Human Computer Interaction (Abstract)

E. Polat , Pennsylvania State University
M. Yeasin , Pennsylvania State University
R. Sharma , Pennsylvania State University
pp. 27

A Structural Approach to Distance Rendering in Personal Auditory Displays (Abstract)

Federico Fontana , University of Verona
Davide Rocchesso , University of Verona
Laura Ottaviani , University of Verona
pp. 33

A Multimodal Electronic Travel Aid Device (Abstract)

A. Fusiello , Università degli Studi di Verona
A. Panuccio , Università degli Studi di Verona
V. Murino , Università degli Studi di Verona
F. Fontana , Università degli Studi di Verona
D. Rocchesso , Università degli Studi di Verona
pp. 39
Speech Generation and Recognition

Lecture and Presentation Tracking in an Intelligent Meeting Room (Abstract)

Ivica Rogina , Universität Karlsruhe
Thomas Schaaf , Universität Karlsruhe
pp. 47

Parallel Computing-Based Architecture for Mixed-Initiative Spoken Dialogue (Abstract)

Ryuta Taguma , Tokyo Institute of Technology
Tatsuhiro Moriyama , Tokyo Institute of Technology
Koji Iwano , Tokyo Institute of Technology
Sadaoki Furui , Tokyo Institute of Technology
pp. 53

3-D N-Best Search for Simultaneous Recognition of Distant-Talking Speech of Multiple Talkers (Abstract)

Satoshi Nakamura , ATR Spoken Language Translation Research Laboratories
Panikos Heracleous , ATR Spoken Language Translation Research Laboratories
pp. 59

Integration of Tone Related Feature for Chinese Speech Recognition (Abstract)

Pui-Fung Wong , Hong Kong University of Science and Technology
Man-Hung Siu , Hong Kong University of Science and Technology
pp. 64

Talking Heads: Which Matching between Faces and Synthetic Voices? (Abstract)

Marc Mersiol , France Télécom R&D
Noël Chateau , France Télécom R&D
Valérie Maffiolo , France Télécom R&D
pp. 69

Robust Noisy Speech Recognition with Adaptive Frequency Bank Selection (Abstract)

Ye Tian , Tsinghua University
Ji Wu , Tsinghua University
Zuoying Wang , Tsinghua University
Dajin Lu , Tsinghua University
pp. 75

Covariance-Tied Clustering Method In Speaker Identification (Abstract)

ZhiQiang Wang , Chinese Academy of Science
Yang Liu , Chinese Academy of Science
Peng Ding , Chinese Academy of Science
Xu Bo , Chinese Academy of Science
pp. 81
Dialogue Processing and Language Understanding

Context-Based Multimodal Input Understanding in Conversational Systems (Abstract)

Joyce Chai , IBM T.J. Watson Research Center
Shimei Pan , IBM T.J. Watson Research Center
Michelle X. Zhou , IBM T.J. Watson Research Center
Keith Houck , IBM T.J. Watson Research Center
pp. 87

Context-Sensitive Help for Multimodal Dialogue (Abstract)

Helen Wright Hastie , AT&T Labs - Research
Michael Johnston , AT&T Labs - Research
Patrick Ehlen , AT&T Labs - Research
pp. 93

Referring to Objects with Spoken and Haptic Modalities (Abstract)

Frédéric Landragin , LORIA Laboratory — France
Nadia Bellalem , LORIA Laboratory — France
Laurent Romary , LORIA Laboratory — France
pp. 99

Towards Visually-Grounded Spoken Language Acquisition (Abstract)

Deb Roy , Massachusetts Institute of Technology
pp. 105

Modeling Output in the EMBASSI Multimodal Dialog System (Abstract)

Christian Elting , European Media Laboratory GmbH
Gregor Möhler , Sony International (Europe) GmbH
pp. 111

Multimodal Dialogue Systems for Interactive TVApplications (Abstract)

Aseel Ibrahim , Nokia Home Communications and Link?ping University
Pontus Johansson , Link?ping University
pp. 117

Viewing and Analyzing Multimodal Human-computer Tutorial Dialogue: A Database Approach (Abstract)

Jack Mostow , Carnegie Mellon University
Joseph Beck , Carnegie Mellon University
Raghu Chalasani , Carnegie Mellon University
Andrew Cuneo , Carnegie Mellon University
Peng Jia , Carnegie Mellon University
pp. 129

Adaptive Dialog Based upon Multimodal Language Acquisition (Abstract)

Sorin Dusan , Rutgers University
James Flanagan , Rutgers University
pp. 135

Integrating Emotional Cues into a Framework for Dialogue Management (Abstract)

Hartwig Holzapfel , University of Karlsruhe
Christian Fuegen , University of Karlsruhe
Matthias Denecke , Carnegie Mellon University
Alex Waibel , Carnegie Mellon University
pp. 141
Signing, Gesturing, and Writing

Gesture Patterns during Speech Repairs (Abstract)

Lei Chen , Purdue University
Mary Harper , Purdue University
Francis Quek , Wright State University
pp. 155

Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures (Abstract)

Sanshzar Kettebekov , Pennsylvania State University
Mohammed Yeasin , Pennsylvania State University
Rajeev Sharma , Pennsylvania State University
pp. 161

Purdue RVL-SLLL ASL Database for Automatic Recognition of American Sign Language (Abstract)

Aleix M. Martínez , Ohio State University and Purdue University
Ronnie B. Wilbur , Purdue University
Robin Shay , Purdue University
Avi C. Kak , Purdue University
pp. 167

Hand Gesture Symmetric Behavior Detection and Analysis in Natural Conversation (Abstract)

Yingen Xiong , Wright State University
Francis Quek , Wright State University
David McNeill , University of Chicago
pp. 179

A Multi-Class Pattern Recognition System for Practical Finger Spelling Translation (Abstract)

Jose L. Hernandez-Rebollar , George Washington University
Robert W. Lindeman , George Washington University
Nicholas Kyriakopoulos , George Washington University
pp. 185

A Map-Based System Using Speech and 3D Gestures for Pervasive Computing (Abstract)

Andrea Corradini , Oregon Health & Science University
Richard M. Wesson , Oregon Health & Science University
Philip R. Cohen , Oregon Health & Science University
pp. 191

Hand Tracking Using Spatial Gesture Modeling and Visual Feedback for a Virtual DJ System (Abstract)

Edward Lin , Carnegie Mellon University
Andy Cassidy , Carnegie Mellon University
Dan Hook , Carnegie Mellon University
Avinash Baliga , Carnegie Mellon University
Tsuhan Chen , Carnegie Mellon University
pp. 197
Translation and Multilingual Interfaces

A PDA-Based Sign Translator (Abstract)

Jing Zhang , Mobile Technologies, LLC
Xilin Chen , Carnegie Mellon University
Jie Yang , Carnegie Mellon University
Alex Waibel , Mobile Technologies, LLC and Carnegie Mellon University
pp. 217

The NESPOLE! Multimodal Interface for Cross-lingual Communication - Experience and Lessons Learned (Abstract)

Loredana Taddei , Aethra Telecomunicazioni
Erica Costantini , University of Trieste
Alon Lavie , Carnegie Mellon University
pp. 223

Research of Machine Learning Method for Specific Information Recognition on the Internet (Abstract)

Dequan Zheng , Harbin Institute of Technology
Yi Hu , Harbin Institute of Technology
Tiejun Zhao , Harbin Institute of Technology
Hao Yu , Harbin Institute of Technology
Sheng Li , Harbin Institute of Technology
pp. 229

Multi-Modal Translation System and Its Evaluation (Abstract)

Shigeo Morishima , Seikei University and ATR, Spoken Language Translation Research Laboratories
Satoshi Nakamura , ATR, Spoken Language Translation Research Laboratories.
pp. 241

Towards Universal Speech Recognition (Abstract)

Zhirong Wang , Carnegie Mellon University
Umut Topkara , Carnegie Mellon University
Tanja Schultz , Carnegie Mellon University
Alex Waibel , Carnegie Mellon University
pp. 247

Improved Named Entity Translation and Bilingual Named Entity Extraction (Abstract)

Fei Huang , Carnegie Mellon University
Stephan Vogel , Carnegie Mellon University
pp. 253
Gaze Tracking and Lipreading

Active Gaze Tracking for Human-Robot Interaction (Abstract)

Rowel Atienza , Australian National University
Alexander Zelinsky , Australian National University
pp. 261

3-D Articulated Pose Tracking for Untethered Diectic Reference (Abstract)

David Demirdjian , Massachusetts Institute of Technology
Trevor Darrell , Massachusetts Institute of Technology
pp. 267

Tracking Focus of Attention in Meetings (Abstract)

Rainer Stiefelhagen , Universität Karlsruhe
pp. 273

A Probabilistic Dynamic Contour Model for Accurate and Robust Lip Tracking (Abstract)

Qiang Wang , Tsinghua University
Haizhou Ai , Tsinghua University
Guangyou Xu , Tsinghua University
pp. 281

Attentional Object Spotting by Integrating Multimodal Input (Abstract)

Chen Yu , University of Rochester
Dana H. Ballard , University of Rochester
Shenghuo Zhu , University of Rochester
pp. 287

Lip Tracking for MPEG-4 Facial Animation (Abstract)

Zhilin Wu , Northwestern University
Petar S. Aleksic , Northwestern University
Aggelos K. Katsaggelos , Northwestern University
pp. 293

Multi-Modal Temporal Asynchronicity Modeling by Product HMMs for Robust (Abstract)

Satoshi Nakamura , ATR Spoken Language Translation Research Laboratories
Ken?ichi Kumatani , ATR Spoken Language Translation Research Laboratories and Nara Institute of Science and Technology
Satoshi Tamura , ATR Spoken Language Translation Research Laboratories and Tokyo Institute of Technology
pp. 305
Applications of Multimodal Interfaces

A Multi-Modal Interface for an Interactive Simulated Vascular Reconstruction System (Abstract)

E. V. Zudilova , University of Amsterdam
P. M. A. Sloot , University of Amsterdam
R. G. Belleman , University of Amsterdam
pp. 313

Universal Interfaces to Multimedia Documents (Abstract)

Helen Petrie , City University, London
Wendy Fisher , City University, London
Ine Langer , Harz University of Applied Studies
Gerhard Weber , University of Kiel
Keith Gladstone , Royal National Institute for the Blind
Cathy Rundle , Royal National Institute for the Blind
Liesbeth Pyfers , Pragma, Hoensbroek
pp. 319

A Video Based Interface to Textual Information for the Visually Impaired (Abstract)

Ali Zandifar , University of Maryland at College Park
Ramani Duraiswami , University of Maryland at College Park
Antoine Chahine , University of Maryland at College Park
Larry S. Davis , University of Maryland at College Park
pp. 325

Modular Approach of Multimodal Integration in a Virtual Environment (Abstract)

Rajarathinam Arangarasan , University of Wisconsin - Madison
George N. Phillips Jr. , University of Wisconsin - Madison
pp. 331

Mobile Multi-Modal Data Services for GPRS Phones and Beyond (Abstract)

Georg Niklfeld , ftw. Telecommunications Research
Michael Pucher , ftw. Telecommunications Research
Robert Finan , Mobilkom Austria AG & Co KG
Wolfgang Eckhart , Sonorys Technology GmbH
pp. 337

Flexi-Modal and Multi-Machine User Interfaces (Abstract)

Brad Myers , Carnegie Mellon University
Robert Malkin , Carnegie Mellon University
Michael Bett , Carnegie Mellon University
Alex Waibel , Carnegie Mellon University
Ben Bostwick , Carnegie Mellon University
Robert C. Miller , Carnegie Mellon University
Jie Yang , Carnegie Mellon University
Matthias Denecke , Carnegie Mellon University
Edgar Seemann , Carnegie Mellon University
Jie Zhu , Carnegie Mellon University
Choon Hong Peck , Carnegie Mellon University
Dave Kong , Carnegie Mellon University
Jeffrey Nichols , Carnegie Mellon University
Bill Scherlis , Carnegie Mellon University
pp. 343

A Real-Time Framework for Natural Multimodal Interaction with Large Screen Displays (Abstract)

N. Krahnstoever , Pennsylvania State University
S. Kettebekov , Pennsylvania State University
M. Yeasin , Pennsylvania State University
R. Sharma , Pennsylvania State University
pp. 349

Embarking on Multimodal Interface Design (Abstract)

Anoop K. Sinha , University of California at Berkeley
James A. Landay , University of California at Berkeley
pp. 355

Multi Modal User Interaction in an Automatic Pool Trainer (Abstract)

Lars Bo Larsen , Aalborg University
Morten Damm Jensen , Aalborg University
Wisdom Kobby Vodzi , Aalborg University
pp. 361

Multimodal Contextual Car-Driver Interface (Abstract)

Daniel Siewiorek , Carnegie Mellon University
Asim Smailagic , Carnegie Mellon University
Matthew Hornyak , Carnegie Mellon University
pp. 367
Sensors, Tools, and Platforms for Multimodal Interfaces

Requirements for Automatically Generating Multi-Modal Interfaces for Complex Appliances (Abstract)

Jeffrey Nichols , Carnegie Mellon University
Brad Myers , Carnegie Mellon University
Thomas K. Harris , Carnegie Mellon University
Roni Rosenfeld , Carnegie Mellon University
Stefanie Shriver , Carnegie Mellon University
Michael Higgins , MAYA Design, Inc.
Joseph Hughes , MAYA Design, Inc.
pp. 377

Articulated Model Based People Tracking Using Motion Models (Abstract)

Huazhong Ning , Chinese Academy of Sciences
Liang Wang , Chinese Academy of Sciences
Weiming Hu , Chinese Academy of Sciences
Tieniu Tan , Chinese Academy of Sciences
pp. 383

Audiovisual Arrays for Untethered Spoken Interfaces (Abstract)

Kevin Wilson , Massachusetts Institute of Technology
Vibhav Rangarajan , Massachusetts Institute of Technology
Neal Checka , Massachusetts Institute of Technology
Trevor Darrell , Massachusetts Institute of Technology
pp. 389

Fingerprint Classification by Directional Fields (Abstract)

Sen Wang , Chinese Academy of Sciences
Wei Wei Zhang , Chinese Academy of Sciences
Yang Sheng Wang , Chinese Academy of Sciences
pp. 395

Towards Vision-Based 3-D People Tracking in a Smart Room (Abstract)

Dirk Focken , Universität Karlsruhe
R. Stiefelhagen , Universität Karlsruhe
pp. 400

Using TouchPad Pressure to Detect Negative Affect (Abstract)

Helena M. Mentis , Cornell University
Geri K. Gay , Cornell University
pp. 406

Musically Expressive Doll in Face-to-Face Communication (Abstract)

Tomoko Yonezawa , ATR Media Integration & Communications Research Laboratories
Kenji Mase , ATR Media Information Science Laboratories
pp. 417

Towards Monitoring Human Activities Using an Omnidirectional Camera (Abstract)

Xilin Chen , Carnegie Mellon University
Jie Yang , Carnegie Mellon University
pp. 423

Smart Platform — A Software Infrastructure for Smart Space (SISS) (Abstract)

Weikai Xie , Tsinghua University
Yuanchun Shi , Tsinghua University
Guanyou Xu , Tsinghua University
Yanhua Mao , Tsinghua University
pp. 429
User Studies and System Evaluation

CATCH-2004 Multi-Modal Browser: Overview Description with Usability Analysis (Abstract)

Jan Kleindienst , IBM Voice Technologies & Systems
Ladislav Seredi , IBM Voice Technologies & Systems
Pekka Kapanen , Nokia Research Center
Janne Bergman , Nokia Research Center
pp. 442

Multimodal Interaction During Multiparty Dialogues: Initial Results (Abstract)

Philip R. Cohen , Oregon Health & Science University
Rachel Coulston , Oregon Health & Science University
Kelly Krout , Oregon Health & Science University
pp. 448

Multi-Modal Embodied Agents Scripting (Abstract)

Yasmine Arafa , Imperial College London
Abe Mamdani , Imperial College London
pp. 454

A Methodology for Evaluating Multimodality in a Home Entertainment System (Abstract)

Jason Williams , Sony Advanced Technology Centre
Georg Michelitsch , Sony Advanced Technology Centre
Gregor Moehler , Sony Advanced Technology Centre
Stefan Rapp , Sony Advanced Technology Centre
pp. 460

Body-Based Interfaces (Abstract)

Changseok Cho , Korea Institute of Science and Technology
Huichul Yang , Pohang University of Science and Technology
Gerard J. Kim , Pohang University of Science and Technology
Sung H. Han , Pohang University of Science and Technology
pp. 466

Evaluation of the Command and Control Cube (Abstract)

Jérôme Grosjean , i3D INRIA
Jean-Marie Burkhardt , INRIA - Universite Paris V
Sabine Coquillart , i3D INRIA
Paul Richard , ISTIA - Innovation
pp. 473

Interruptions as Multimodal Outputs: Which are the Less Disruptive? (Abstract)

Ernesto Arroyo , Massachusetts Institute of Technology
Ted Selker , Massachusetts Institute of Technology
Alexandre Stouffs , Massachusetts Institute of Technology
pp. 479

Experimentally Augmenting an Intelligent Tutoring System with Human-Supplied Capabilities: Adding Human-Provided Emotional Scaffolding to an Automated Reading Tutor that Listens (Abstract)

Gregory Aist , Project LISTEN
Barry Kort , Massachusetts Institute of Technology
Rob Reilly , Massachusetts Institute of Technology
Jack Mostow , Project LISTEN
Rosalind Picard , Massachusetts Institute of Technology
pp. 483

Individual Differences in Facial Expression: Stability over Time, Relation to Self-Reported Emotion, and Ability to Inform Person Identification (Abstract)

Jeffrey F. Cohn , University of Pittsburgh and Carnegie Mellon University
Karen Schmidt , University of Pittsburgh
Ralph Gross , Carnegie Mellon University
Paul Ekman , University of California at San Francisco
pp. 491
Face Detection, Recognition, and Synthesis

Training a Talking Head (Abstract)

Michael M. Cohen , University of California at Santa Cruz
Dominic W. Massaro , University of California at Santa Cruz
Rashid Clark , University of California at Santa Cruz
pp. 499

Labial Coarticulation Modeling for Realistic Facial Animation (Abstract)

Piero Cosi , Istituto di Scienze e Tecnologie della Cognizione- C.N.R.
Emanuela Magno Caldognetto , Istituto di Scienze e Tecnologie della Cognizione- C.N.R.
Giulio Perin , Universit? di Padova
Claudio Zmarich , Istituto di Scienze e Tecnologie della Cognizione- C.N.R.
pp. 505

Improved Information Maximization based Face and Facial Feature Detection from Real-time Video and Application in a Multi-Modal Person Identification System (Abstract)

Ziyou Xiong , University of Illinois at Urbana-Champaign
Yunqiang Chen , University of Illinois at Urbana-Champaign
Roy Wang , University of Illinois at Urbana-Champaign
Thomas S. Huang , University of Illinois at Urbana-Champaign
pp. 511

Animating Arbitrary Topology 3D Facial Model Using the MPEG-4 FaceDefTables (Abstract)

Dalong Jiang , Chinese Academy of Sciences
Wen Gao , Chinese Academy of Sciences and Harbin Institute of Technology
Zhiguo Li , Chinese Academy of Sciences
Zhaoqi Wang , Chinese Academy of Sciences
pp. 517

An Improved Active Shape Model for Face Alignment (Abstract)

Wei Wang , Beijing Polytechnic University
Shiguang Shan , Chinese Academy of Sciences
Wen Gao , Chinese Academy of Sciences and Harbin Institute of Technology
Bo Cao , Chinese Academy of Sciences
Baocai Yin , Beijing Polytechnic University
pp. 523

An Improved Algorithm for Hairstyle Dynamics (Abstract)

Wenjun Lao , Beijing Polytechnic University
Dehui Kong , Beijing Polytechnic University
Baocai Yin , Beijing Polytechnic University
pp. 535
Author Index

Author Index (PDF)

pp. 541
81 ms
(Ver 3.3 (11022016))