We present an encoder rate control algorithm based on a perceptual criterion for very low bit rate video. The emerging H.263 recommendation for transmission of video over narrow bandwidth channels supports variable frame rate video streams. This additional freedom enables the rate control algorithm to explore the tradeoff between temporal sampling and spatial quality, while keeping the generated bit rate within the available channel bandwidth. The proposed criterion takes advantage of this tradeoff by using dynamic models of encoder performance and characteristics of the human visual system to adaptively set the nominal quantizer scale factor and instantaneous frame rate. Experiments with typical video-conference sequences demonstrate the application of this new algorithm to both constant and variable bit rate channels.