Image matching plays an important role in visual mapping, a critical task of vision based mobile robot navigation. Based on the observation that the visual content in these video sequences generally changes in a slow and continuous mode, a concept of "video local pattern" is proposed to model each video frame, which is defined as a set of frames that are visuallysimilar and temporally-adjacent to that frame. Instead of manually labelling, a tracking based method is developed to automatically detect the local pattern for each frame. A model is then estimated from a local pattern, and the matching of images is performed by comparing these models. Experimental results demonstrated its improvement over that comparing individual frames.