In this paper we describe ICARUS, an integrated architecture for intelligent physical agents. The framework supports long-term memories for hierarchical concepts and skills, along with mechanisms for recognizing concepts that hold in the environment, determining which skills are applicable, and selecting for execution the skill with the highest expected value. We illustrate these processes with examples from the domain of in-city driving, and we report experimental studies on a package-delivery task that examine ICARUS' ability to combine reactive behavior with persistence over time. We conclude with a discussion of related work on agent architectures and our plans for extending the system.