A two-neural network approach to solving optimal control problems is described in this study. This approach called the adaptive critic method consists of one neural network called the supervisor or critic and a second network called an action network or a controller. The inputs to both these networks are the current states of the system to be controlled. Each network is trained through output of the other network and the conditions for optimal control. When their outputs are mutually consistent, the controller network output is optimal. The optimality is however limited to the underlying model. Hence, we develop a Lyapunov based theory for robust stability of these controllers when there is input uncertainty. We illustrate this approach through a longitudinal autopilot of a nonlinear missile problem.