A road sign (RS) recognition system poses a real challenge for machine vision. It must recognize a wide variety of RS?s under considerable variations in illumination and imaging geometry -all in real-time. Such a system is presented in this paper, with emphasis on the system architecture and the specific model-based techniques used in the different processing steps. Central to this are a unique physics-based color detection approach and a novel template matching scheme for planar objects. Since the approach strongly relies on modelling for both detection and recognition, it offers the advantage of being reconfigurable by changing only few parameters. The system is modular with respect to sensor, and the recognition data structure is simple to extend and maintain, and is easily adaptable to different regulations, e.g. North American vs European RS?s. The data needed for recognition is computed automatically by modelling image formation with a few geometri-cal parameters. Experimental results are presented which demon-strate the performance of the system in a real task environment with high overall performance.