This paper presents an approach for creating realistic shadows of objects composited into a novel background. Existing techniques either do not deal with the shadows or require that the relationship of the light source, reference plane, and camera be the same in the source and target scenes. We observe that scenes containing a ground plane and some up-right objects, e.g. walls, crowds, desks, street lamps, trees, etc., are common in natural environments, and show that two vertical lines and their cast shadows are enough to recover primary light source, which makes it possible to construct shadows of inserted planar objects in the single target image. We also demonstrate a technique to calibrate the camera when two views are available, which enables us to create shadows of general 3D objects in a target scene. The proposed method therefore advances the image based techniques one step further toward the synthesis of the scenes with variation of both lighting and viewpoints. We demonstrate our proposed approach for real scenes.