While current approaches for segmenting or classifying audiovisual data are mostly focused on visual information, audio information may also play an important role in the content parsing for many applications. In this paper, we propose a novel scheme for determining video scenes by analyzing both audio and video data. We also describe some preliminary experiments that we have performed and report some of the results in the automatic segmentation and classification of audiovisual data for video indexing and retrieval. Audio segmentation and classification are used not only for extracting useful video indexing information, but also for performing a preprocessing step for other analysis modules.