A content-aware rate controller (CRC) for MPEG-4 FGS stored video streaming over the Internet is proposed. The CRC optimally uses available bandwidth and client buffer to produce smooth video quality and provide protection to video segments with important content. The FGS video stream is forward-shifted to client buffer by actively dropping high enhancement layers. When bandwidth decreases sharply, the FS buffer is used as bandwidth so that the sharp bandwidth drop is hidden from the decoder. The active dropping and forward-shifting process is restarted when bandwidth recovers. Thus, higher layers of less importance content are dropped and lower layers of more importance are protected. A priori information about the stream content is used in the CRC such that video segments with important content are protected by allocating them more bandwidth.