We propose an approach to object manipulation. In the approach, the user?s hand or fingers are represented by a set of interface points, the interaction force on each interface point is computed by the collision response computation algorithm, and the motion of the object is simulated based on the interaction force. We employed a fast collision response computation method to accommodate a large number of interaction points. Also, we devised a method for stable computation of object motion. We carried out experiments on interaction with models of varying complexity and demonstrated the feasibility of our approach for dexterous manipulations.