Advances in digital signal processing technology make it possible to embed the low-level functions performed by a video surveillance system into cameras. The result is metadata cameras, like MPEG-7 cameras, that provide descriptions of what they see in the form of XML documents served over a network. We use this concept to build video information servers that are conceptually similar to MPEG-7 cameras, but differ in that they interact with client applications and can be configured dynamically to perform different functions. We demonstrate the use of these video information servers in some simple surveillance applications. Dynamic configuration of the servers allows them to do more than describe video content. For example, we present a functioning surveillance system, built with content description servers, that describes the activity in a hockey game based on the observations of several cameras. Thus, we demonstrate that content description servers provide building blocks with which to create networked video surveillance systems.