A prerequisite to calibrated camera pose estimation is the construction of a camera neighborhood adjacency graph, a connected graph defining the pose neighbors of the camera set. Pose neighbors to a camera C are images containing sufficient overlap in image content with the image from C that they can be used to correctly estimate the pose of C using structure-from-motion techniques. In a video stream, the camera neighborhood adjacency graph is often a simple connected path; frame poses are only estimated relative to their immediate neighbors.

We propose a novel method to build more complex camera adjacency graphs that are suitable for estimating the pose of large numbers of wide- and narrow-baseline images. We employ Content-Based Image Retrieval techniques to identify similar images likely to be graph neighbors. We also develop an optimization to improve graph accuracy that is based on an observation of common camera motions taken when photographing with the intent of structure-from-motion. Our results substantiate the use of our method for determining neighbors for pose estimation.