The objective of this paper is to provide a personalized multi-sensory learning model with mobile handheld learning devices to enhance pervasive learning. This paper distinguishes itself by solving several challenges that the current mobile learning research is facing and unsolved. Our research consists of three phases: Phase one is to develop an ontology and agent based knowledge network framework with content management mechanism for learning content structure, description, representation, retrieval, reuse, revise, retain, exchange and sharing. Phase two is to develop intelligent mobile handheld learning devices with universal access mechanism with personalized learning experience for adaptive and seamless pervasive learning. Phase three is to develop a multi-sensory learning system with compelling examples of pen based, annotation based, context driven, and location based learning service and applications for pervasive learning.