Obtaining a digital model of a real-world 3D scene is a challenging task pursued by computer vision and computer graphics. Given an initial approximate 3D model, a popular refinement process is to perform a bundle adjustment of the estimated camera position, camera orientation, and scene points. Unfortunately, simultaneously solving for both camera position and camera orientation is an ill-conditioned problem. To address this issue, we propose an improved, camera-orientation independent cost function that can be used instead of the standard bundle adjustment cost function. This yields a new bundle adjustment formulation which exhibits noticeably better numerical behavior, but at the expense of an increased computational cost. We alleviate the additional cost by automatically partitioning the dataset into smaller subsets. Minimizing our cost function for these subsets still achieves significant error reduction over standard bundle adjustment. We empirically demonstrate our formulation using several different size models and image sequences.