This Article 
 Bibliographic References 
 Add to: 
Visual Input for Pen-Based Computers
March 2002 (vol. 24 no. 3)
pp. 313-328

The design and implementation of a camera-based, human-computer interface for acquisition of handwriting is presented. The camera focuses on a standard sheet of paper and images a common pen; the trajectory of the tip of the pen is tracked and the contact with the paper is detected. The recovered trajectory is shown to have sufficient spatio-temporal resolution and accuracy to enable handwritten character recognition. More than 100 subjects have used the system and have provided a large and heterogeneous set of examples showing that the system is both convenient and accurate.

[1] B.D. Anderson and J.B. Moore, Optimal Filtering. Prentice Hall, Inc., 1979.
[2] H. Bunke, T. Von Siebenhal, T. Yamasaki, and M. Schenkel, “On Line Handwriting Data Acquisition Using a Video Camera,” Proc. Int'l Conf. Document Analysis and Recognition, pp. 573-576, 1999.
[3] J. Canny, “A Computational Approach to Edge Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 679-698, June 1986.
[4] J.L. Crowley, “Vision for Man-Machine Interaction,” Robotics and Autonomous Systems, vol. 19, nos. 3-4, pp. 347-358, 1997.
[5] J.L. Crowley, F. Bernard, and J. Coutaz, “Finger Tracking as an Input Device for Augmented Reality,” Proc. Int'l Workshop Face and Gesture Recognition, pp. 195-200, 1995.
[6] S. Elrod et al., "Liveboard: A Large Interactive Display Supporting Group Meetings, Presentations, and Collaborations," Proc. 1992 Conf. Human Factors in Computing Systems (CHI 92), ACM Press, New York, 1992, pp. 599-607.
[7] G.D. Forney, “The Viterbi Algorithm,” Proc. IEEE, vol. 61, no. 3, pp. 268-278, 1973.
[8] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[9] A. Gelb, Applied Optimal Estimation. The MIT Press, 1974.
[10] S. Impedovo, L. Ottaviano, and S. Occhinegro, “Optical Character Recognition—A Survey,” Int'l J. Pattern Recognition and Artificial Intelligence, vol. 5, pp. 1-24, 1991.
[11] R.E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” Trans. ASME J. Basic Eng., pp. 35-45, 1960.
[12] G. Lorette and R. Plamondon, “Dynamic Approaches to Handwritten Signature Verification,” Computer Processing of Handwriting, pp. 21-47, 1990.
[13] B.D. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proc. Seventh Int'l Joint Conf. Artificial Intelligence, 1981.
[14] M.E. Munich, “Visual Input for Pen-Based Computers,” PhD thesis, Calif. Inst. of Technology, Pasadena, Jan. 2000. .
[15] M.E. Munich and P. Perona, “Visual Input for Pen-Based Computers,” Proc. 13th Int'l Conf. Pattern Recognition, 1996.
[16] M.E. Munich and P. Perona, “Camera-Based ID Verification by Signature Tracking,” Proc. Fifth European Conf. Computer Vision, H. Burkhardt and B. Neumann, eds., pp. 782-796, 1998.
[17] M.E. Munich and P. Perona, “Visual Signature Verification Using Affine Arc-Length,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 180-186, 1999.
[18] S. Nabeshima, S. Yamamoto, K. Agusa, and T. Taguchi, “Memo-Pen: A New Input Device,” Proc. Int'l Conf. Human Factors in Computing Systems (CHI), pp. 256-257, 1995.
[19] M. Nakagawa, T. Oguni, T. Yoshino, K. Horiba, and S. Sawada, “Interactive Dynamic Whiteboard for Educational Applications,” Proc. Int'l Conf. Virtual Systems and Multimedia, pp. 479-484, 1996.
[20] R. Plamondon and B. Clément, “Dependence of Peripheral and Central Parameters Describing Handwriting Generation on Movement Direction,” Human Movement Science, vol. 10, pp. 193-221, 1991.
[21] R. Plamondon and F.J. Maarse, “An Evaluation of Motor Models of Handwriting,” IEEE Trans. Systems, Man, and Cybernetics, vol. 19, no. 5, pp. 1,060-1,072, 1989.
[22] R. Plamondon and S.N. Srihari, On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 63-84, Jan. 2000.
[23] L.R. Rabiner and B.H. Juang, Fundamentals of Speech Recognition, Prentice Hall, Upper Saddle River, N.J., 1993.
[24] B. Simard, B. Prasad, and M.K. Sinha, “On-Line Character Recognition Using Handwriting Modelling,” Pattern Recognition, vol. 26, no. 7, pp. 993-1007, 1993.
[25] C.C. Tappert, C.Y. Suen, and T. Wakahara, “The State of the Art in On-Line Handwriting Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 8, pp. 179-190, Aug. 1990.
[26] H.L. Van Trees, Detection, Estimation, and Modulation Theory. New York: Wiley, 1968.
[27] P. Viviani and G. McCollum, “The Relation between Linear Extent and Velocity in Drawings Movements,” Neuroscience, vol. 10, no. 1, pp. 211-218, 1983.
[28] P. Viviani and C. Terzuolo, “Trajectory Determines Movement Dynamics,” Neuroscience, vol. 7, no. 2, pp. 431-437, 1982.
[29] J. Vredenbregt and W.G. Koster, “Analysis and Synthesis of Handwriting,” Philips Techical Rev. vol. 32, no. 3/4, pp. 73-78, 1971.
[30] P.D. Wellner, “Adaptative Thresholding for the Digitaldesk,” Technical Report EPC-1993-110, 1993.
[31] P.D. Wellner, “Self Calibration for Digitaldesk,” Technical Report EPC-1993-109, 1993.
[32] T. Yamasaki and T. Hattori, “A New Data Tablet System for Handwriting Characters and Drawing Based on Image Processing,” Proc. IEEE Int'l Conf. Systems, Man and Cybernetics, pp. 428-431, 1996.

Index Terms:
systems and applications, active and real-time vision, pen-based computing, pen-based interface
M.E. Munich, P. Perona, "Visual Input for Pen-Based Computers," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 313-328, March 2002, doi:10.1109/34.990134
Usage of this product signifies your acceptance of the Terms of Use.