This paper presents a novel method for detecting vehicles as obstacles in various road scenes using a single on-board camera. Vehicles are detected by testing whether the motion of a set of three horizontal line segments, which are always on the vehicles, satisfies the motion constraint of the ground plane or that of the surface plane of the vehicles. The motion constraint of each plane is derived from the projective invariant combined with the vanishing line of the plane that is a prior knowledge of road scenes. The proposed method is implemented into a newly developed on-board LSI. Experimental results for real road scenes under various conditions show the effectiveness of the proposed method.