STAVIES: A System for Information Extraction from Unknown Web Data Sources through Automatic Web Wrapper Generation Using Clustering Techniques
Issue No.12 - December (2005 vol.17)
Konstantinos Raftopoulos , IEEE Computer Society
Theodora A. Varvarigou , IEEE
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2005.203
A fully automated wrapper for information extraction from Web pages is presented. The motivation behind such systems lies in the emerging need for going beyond the concept of "human browsing.” The World Wide Web is today the main "all kind of information” repository and has been so far very successful in disseminating information to humans. By automating the process of information retrieval, further utilization by targeted applications is enabled. The key idea in our novel system is to exploit the format of the Web pages to discover the underlying structure in order to finally infer and extract pieces of information from the Web page. Our system first identifies the section of the Web page that contains the information to be extracted and then extracts it by using clustering techniques and other tools of statistical origin. STAVIES can operate without human intervention and does not require any training. The main innovation and contribution of the proposed system consists of introducing a signal-wise treatment of the tag structural hierarchy and using hierarchical clustering techniques to segment the Web pages. The importance of such a treatment is significant since it permits abstracting away from the raw tag-manipulating approach. Experimental results and comparisons with other state of the art systems are presented and discussed in the paper, indicating the high performance of the proposed algorithm.
Index Terms- Automatic wrappers, generic wrappers, data source wrappers, Web mining, Web data extraction, Web structure mining, intelligent agents on the Web, resource discovery, information retrieval.
Nikolaos K. Papadakis, Dimitrios Skoutas, Konstantinos Raftopoulos, Theodora A. Varvarigou, "STAVIES: A System for Information Extraction from Unknown Web Data Sources through Automatic Web Wrapper Generation Using Clustering Techniques", IEEE Transactions on Knowledge & Data Engineering, vol.17, no. 12, pp. 1638-1652, December 2005, doi:10.1109/TKDE.2005.203