In this paper, we propose a robust method to register a panoramic range data obtained by a panoramic laser range finder and an omni-directional image acquired by a stereo omni-directional system (SOS), by matching the horizontal and vertical edge histograms of the two images. The proposed method first estimates the relative horizontal rotation angle by matching the vertical edge histograms. Next, for each horizontal direction, we generate horizontal edge histograms for both SceneModeler and SOS, using the perspective images with a FOV of 90° centered at the concerned direction. Then we match the horizontal edge histogram in each direction and use the matching results to estimate the relative pose between the sensors. Experimental results show the effectiveness and robustness of the proposed method.