The purpose of audio signal discrimination is needed to build two different libraries: speech library and music library, from a stream of sounds. The classification algorithms of audio signal can be divided into three categories: time-domain, frequency-domain, and time-frequency domain approaches. The first two approaches were discussed thoroughly in literature. Since audio signals are considered nonstationary, in general, these two approaches may fail in reflecting the behavior of each signal correctly. Therefore, in this paper, a joint distribution approach, the evolutionary spectrum (ES), is introduced and proposed for the discrimination purpose. To show the effectiveness of the proposed method, some comparison examples with classical methods is done.