Interactive 3D modeling is the process of building a 3D model of an object or a scene in real-time while the 3D (range) data is acquired. This is possible only if the computational complexity of all involved algorithms is linear with respect to the amount of data. We propose a new framework for 3D modeling where a complete modeling chain meets with this requirement. The framework is based on the use of vector fields as an implicit surface representation. Each modeling step, registration, surface reconstruction, geometric fusion, compression and visualization is solved and explained using the vector fields without anyintermediate representations. The proposed framework allows model reconstruction from any type of 3D data, surface patches, curves, unorganized sets of points or a combination of these.