This paper is concerned with the problem of dynamically reconstructing the 3D surface of an object undergoing non-rigid motion. The problem is cast as reconstructing a continuous optimal 3D hyper-surface in 4D space-time from a set of calibrated video sequences. The imaging model of video cameras in 4D space-time is derived and a photo-inconsistency cost function is defined for a hyper-surface in the 4D space-time. We use a 4D node-cut algorithm to find a global minimum of the cost function and obtain the corresponding optimal hyper-surface. Experimental results show that the proposed algorithm is effective in recovering continuously changing shapes and exhibits good noise resistance.