In this paper we describe a system for physical animation of rigid and deformable objects. These are represented as groups of particles linked by linear constraints, while a Verlet integrator is used for motion computation. Unlike traditional approaches, we accomplish physical simulation without explicitly computing orientation matrices, torques or inertia tensors. The main contribution of our work is related to the way collisions are handled by the system, which employs different approaches for deformable and rigid bodies. In particular, we show how collision detection using the GJK algorithm [9] and bounding sphere hierarchies can be combined with the projection based collision response technique described by Jakobsen [14].