In a two-year study of a collaborative human-robot system, researchers observed a science team in Pittsburgh and a robot in Chile. The system was part of a project intended to inform planetary exploration while studying a terrestrial desert. Over two years, autonomy increased such that the robot could execute sequences of commands and, under certain circumstances, make autonomous decisions about instrument deployment and data collection. Analysis of the observational and project artifact data addressed the impact of increasing levels of autonomy on how users reach common ground with a remote robot in terms of an accurate, shared understanding of the robot's context, planning, and actions. Findings suggest that as autonomy increases, users' inability to understand the reasons for the robot's actions disrupts the creation of common ground. The authors describe the implications of this work for human-robot system design. This article is part of a special issue on Interacting with Autonomy.