This paper describes a framework for collaboration between a user and a multi-agent system to achieve adjustable autonomy. Adjustable autonomy (AA) is when the levels of autonomy of the agent system - its control over its reasoning - changes during execution due to interaction with a user or other systems. We describe a prototype agent development environment that allows users flexible on-line control over an otherwise completely autonomous agent system.AA can improve productivity during system design, allow earlier deployment, and create a more flexible system. AA can reduce the load on instructors when used in training situations, and allow the user to aid the system when faced with unforseen situations or problems. Examples are given of AA in simulated pilots for fighter aircraft. We found that the AA - the user's ability to modify the behavior of the agents during execution, resulted in a better, more flexible system.