The simulation of fracture leads to collision-intensive situations that call for efficient collision detection algorithms and data structures. Bounding volume hierarchies (BVHs) are a popular approach for accelerating collision detection, but they rarely see application in fracture simulations, due to the dynamic creation and deletion of geometric primitives. We propose the use of balanced trees for storing BVHs, as well as novel algorithms for dynamically restructuring them in the presence of progressive or instantaneous fracture. By paying a small loss of fitting quality compared with complete reconstruction, we achieve more than one order of magnitude speedup in the update of BVHs.