Adaptive Critic Designs (ADC) is sometimes called Reinforcement Learning Systems (RLS), Approximate Dynamic Programming (ADP) or Neurodynamic Programming. They are the only class of well-grounded engineering design, from any type of learning research, which address the general problem of how to optimize a measure of utility or goal satisfaction, over multiple time periods into the future, in an unknown noisy, nonlinear environment. Applications to cars and missiles begin to confirm their advantages over older methods and their ability to overcome the “curse of dimensionality” in dynamic programming. Some ACDs can be formulated as new designs for “adaptive control,” which - unlike classical neuro-control designs - offer universal stability and fast response to transient disturbances. I have argued that functional understanding of intelligence in mammal brains requires a model of the brain as an ACD. This paper reviews this field, and the new research needed to replicate mammal-like intelligence and beyond.