The quality and stability of noise shaping is a concern in the design of higher-order delta-sigma modulators for high-resolution, high-speed oversampled analog-to-digital conversion. We reformulate noise-shaping modulation alternatively as a nonlinear optimal control problem, where the objective is to find the binary modulation sequence that minimizes signal swing in a cascade of integrators operating on the difference between the input signal and the modulation sequence. We use reinforcement learning to adaptively optimize a nonlinear neural classifier, which outputs modulation bits from the values of the input signal and integration state variables. Analogous to the classical pole balancing control problem, a punishment signal triggers learning whenever any of the integrators saturate. We train a simple classifier consisting of locally tuned, binary address-encoded neurons to produce stable noise shaping modulation, and present experimental results obtained from analog VLSI modulators of orders one and two. The integrated classifier contains an array of 64 neurons trained on-chip with a simplified variant on reinforcement learning.