Issue No.05 - May (2011 vol.23)
Pin-Chieh Sung , National Taiwan University, Taipei
Chi-Yao Tseng , National Taiwan University and Academia Sinica, Taipei
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2010.147
E-mail communication is indispensable nowadays, but the e-mail spam problem continues growing drastically. In recent years, the notion of collaborative spam filtering with near-duplicate similarity matching scheme has been widely discussed. The primary idea of the similarity matching scheme for spam detection is to maintain a known spam database, formed by user feedback, to block subsequent near-duplicate spams. On purpose of achieving efficient similarity matching and reducing storage utilization, prior works mainly represent each e-mail by a succinct abstraction derived from e-mail content text. However, these abstractions of e-mails cannot fully catch the evolving nature of spams, and are thus not effective enough in near-duplicate detection. In this paper, we propose a novel e-mail abstraction scheme, which considers e-mail layout structure to represent e-mails. We present a procedure to generate the e-mail abstraction using HTML content in e-mail, and this newly devised abstraction can more effectively capture the near-duplicate phenomenon of spams. Moreover, we design a complete spam detection system Cosdes (standing for COllaborative Spam DEtection System), which possesses an efficient near-duplicate matching scheme and a progressive update scheme. The progressive update scheme enables system Cosdes to keep the most up-to-date information for near-duplicate detection. We evaluate Cosdes on a live data set collected from a real e-mail server and show that our system outperforms the prior approaches in detection results and is applicable to the real world.
Spam detection, e-mail abstraction, near-duplicate matching.
Pin-Chieh Sung, Chi-Yao Tseng, "Cosdes: A Collaborative Spam Detection System with a Novel E-Mail Abstraction Scheme", IEEE Transactions on Knowledge & Data Engineering, vol.23, no. 5, pp. 669-682, May 2011, doi:10.1109/TKDE.2010.147