Recent research in projector-camera systems has overcome many of the obstacles to deploying and using intelligent displays for a wide range of applications. In parallel with these developments, projector costs continue to decline with corresponding increase in resolution, brightness and contrast ratio. In light of this trend, we are exploring the unique capabilities that camera-projector systems can offer to intelligent environments and ubiqutous computing. Our initial step towards environments that are intelligently augmented by projector-camera devices, is a smart bookshelf application. The system utilizes a camera pair and a projector to monitor the state of a real world library shelf. As books are added to the shelf a foreground detection algorithm which takes into account the projected information yields new pixels in each view that are then verfied using a planar parallax constraint across both cameras to yield the book spine. Using a simple calibration scheme, the homography induced by the world plane in which book spines approximately lie is estimated. Users are then able to query for the presence of a book through a user interface and book spines are highlighted by transforming image pixels to their corresponding points in the projector's frame via the known homography. The system also can display the state of the bookshelf at any time in the past. Projected information can also be used to enhance the image-processing tasks at hand and we briefly explore this in this work.