A smart camera combines video sensing, high-level video processing and communication within a single embedded device. Such cameras are key components in novel surveillance systems. This paper reports on a prototyping development of a smart camera for traffic surveillance. We present its scalable architecture comprised of a CMOS sensor, digital signal processors (DSP), and a network processor. We further discuss the mapping of high-level video processing algorithms to embedded DSP-based platforms and identify typical pitfalls for the porting of software from desktops to embedded platforms. Our mapping strategies are demonstrated on an algorithm for automatic detection of stationary vehicles. This algorithm is migrated from a Matlab-based prototyping implementation to an embedded DSP implementation in our smart camera. Our implemented smart camera prototype streams the video data over an IP-network to a central monitoring station and is able to detect stationary vehicles and blocking cargo on highways within the required real-time constraints of six seconds.