3D scene reconstruction from 2D stereo pairs or sequences has been an interesting issue in computer vision for years. So far, most research has used images acquired by linear low-distortion lenses that can be modeled by pinhole mapping. In recent years, the fisheye lens has received increasing attention and use due to its extremely wide field of view (FOV). However, little has been published on stereo reconstruction of fisheye images. In this paper, we develop an algorithm to compute 3D structure of feature points from correspondences in fisheye pairs, and identify the problems remaining to be solved. Test results on synthetic data show the efficacy of our algorithm.