This paper presents a real-time motion capture system of 3D multi-part objects, whose purpose is to do seamless mapping objects in the real world into virtual environments easily. In general, virtual environment applications such as man-machine seamless interaction require the system to estimate accurate motion parameters at real-time for natural objects such as human bodies. To achieve this requirement, we have been developing a vision-based motion capture system which reconstructs time-varying motion parameters of 3-D multi-part objects. Advantage of such vision-based system is that it is possible to acquire the other scene parameters such as shape and surface properties at the same time, using the same equipment in measuring motion. In this paper, as our first system, we have implemented a color-marker based motion capture system which realizes multi-view fusion and have demonstrated our motion capture and reconstruction system works at real-time on PC-cluster.