This paper presents new results in the field of video coding and compression using 3D information. Contrary to prior art in 3D model-based coding where 3D models have to be known, the 3D models are here automatically computed from the original video sequence. The camera parameters and the scene content are supposed unknown and we only assume a static scene viewed by a moving monocular camera. The video sequence is then processed on the fly. A stream of 3D models is extracted and compressed, using adapted computer vision and compression techniques. The visualization of the reconstructed video sequence uses an adapted morphing method. We finally show results obtained with the proposed compression scheme, compared with the H26L compression standard. We demonstrate the efficiency of our approach, especially in the field of very low bitrate coding.