This paper will discuss learning in hybrid models that goes beyond simple rule extraction from backpropagation networks. Although simple rule extraction has received a lot of research attention, to further develop hybrid-learning models that include both symbolic and sub-symbolic knowledge and that learn autonomously, it is necessary to study autonomous learning of both sub-symbolic and symbolic knowledge in integrated architectures. This paper will describe knowledge extraction from neural reinforcement learning. It includes two approaches to wards extracting plan knowledge: the extraction of explicit, symbolic rules from neural reinforcement learning, and the extraction of complete plans. This work points to the creation of a general framework for achieving the sub-symbolic to symbolic transition in an integrated autonomous learning framework.