The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.06 - June (2010 vol.22)
pp: 770-783
Bin Cao , Hong Kong University of Science and Technology, Hong Kong
Evan Wei Xiang , Hong Kong University of Science and Technology, Hong Kong
Qiang Yang , Hong Kong University of Science and Technology, Hong Kong
ABSTRACT
A major problem of classification learning is the lack of ground-truth labeled data. It is usually expensive to label new data instances for training a model. To solve this problem, domain adaptation in transfer learning has been proposed to classify target domain data by using some other source domain data, even when the data may have different distributions. However, domain adaptation may not work well when the differences between the source and target domains are large. In this paper, we design a novel transfer learning approach, called BIG (Bridging Information Gap), to effectively extract useful knowledge in a worldwide knowledge base, which is then used to link the source and target domains for improving the classification performance. BIG works when the source and target domains share the same feature space but different underlying data distributions. Using the auxiliary source data, we can extract a “bridge” that allows cross-domain text classification problems to be solved using standard semisupervised learning algorithms. A major contribution of our work is that with BIG, a large amount of worldwide knowledge can be easily adapted and used for learning in the target domain. We conduct experiments on several real-world cross-domain text classification tasks and demonstrate that our proposed approach can outperform several existing domain adaptation approaches significantly.
INDEX TERMS
Data mining, transfer learning, cross-domain, text classification, Wikipedia.
CITATION
Bin Cao, Evan Wei Xiang, Qiang Yang, "Bridging Domains Using World Wide Knowledge for Transfer Learning", IEEE Transactions on Knowledge & Data Engineering, vol.22, no. 6, pp. 770-783, June 2010, doi:10.1109/TKDE.2010.31
REFERENCES
[1] W. Dai, Q. Yang, G.-R. Xue, and Y. Yu, "Boosting for Transfer Learning," Proc. 24th Ann. Int'l Conf. Machine Learning (ICML '07), pp. 193-200, June 2007.
[2] J. Jiang and C. Zhai, "Instance Weighting for Domain Adaptation in NLP," Proc. 45th Ann. Meeting of the Assoc. for Computational Linguistics (ACL '07), June 2007.
[3] G.-R. Xue, W. Dai, Q. Yang, and Y. Yu, "Topic-Bridged PLSA for Cross-Domain Text Classification," Proc. 31st Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval (SIGIR '08), pp. 627-634, July 2008.
[4] A. Argyriou, C.A. Micchelli, M. Pontil, and Y. Ying, "A Spectral Regularization Framework for Multi-Task Structure Learning," Proc. 21st Ann. Conf. Neural Information Processing Systems (NIPS '07), Dec. 2007.
[5] R. Raina, A. Battle, H. Lee, B. Packer, and A.Y. Ng, "Self-Taught Learning: Transfer Learning from Unlabeled Data," Proc. 24th Ann. Int'l Conf. Machine Learning (ICML '07), pp. 759-766, June 2007.
[6] J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. Wortman, "Learning Bounds for Domain Adaptation," Proc. 21st Ann. Conf. Neural Information Processing Systems (NIPS '07), Dec. 2007.
[7] S.J. Pan and Q. Yang, "A Survey on Transfer Learning," IEEE Trans. Knowledge and Data Eng., preprint, 12 Oct. 2009, doi: 10.1109/TKDE.2009.191.
[8] S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira, "Analysis of Representations for Domain Adaptation," Proc. 20th Ann. Conf. Neural Information Processing Systems (NIPS '06), pp. 137-144, Dec. 2006.
[9] W. Dai, G.-R. Xue, Q. Yang, and Y. Yu, "Co-Clustering Based Classification for Out-of-Domain Documents," Proc. 13th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining (KDD '07), pp. 210-219, Aug. 2007.
[10] B. Zadrozny, "Learning and Evaluating Classifiers Under Sample Selection Bias," Proc. 21th Ann. Int'l Conf. Machine Learning (ICML '04), p. 114, July 2004.
[11] H.D. III, "Frustratingly Easy Domain Adaptation," Proc. 45th Ann. Meeting of the Assoc. for Computational Linguistics (ACL '07), June 2007.
[12] J. Blitzer, M. Dredze, and F. Pereira, "Biographies, Bollywood, Boom-Boxes and Blenders: Domain Adaptation for Sentiment Classification," Proc. 45th Ann. Meeting of the Assoc. for Computational Linguistics (ACL '07), pp. 440-447, June 2007.
[13] S.I. Lee, V. Chatalbashev, D. Vickrey, and D. Koller, "Learning a Meta-Level Prior for Feature Relevance from Multiple Related Tasks," Proc. 24th Ann. Int'l Conf. Machine Learning (ICML '07), pp. 489-496, June 2007.
[14] E. Gabrilovich and S. Markovitch, "Feature Generation for Text Categorization Using World Knowledge," Proc. 19th Int'l Joint Conf. Artificial Intelligence (IJCAI '05), pp. 1048-1053, July/Aug. 2005.
[15] E. Gabrilovich and S. Markovitch, "Harnessing the Expertise of 70,000 Human Editors: Knowledge-Based Feature Generation for Text Categorization," J. Machine Learning Research, vol. 8, pp. 2297-2345, 2007.
[16] E. Gabrilovich and S. Markovitch, "Overcoming the Brittleness Bottleneck Using Wikipedia: Enhancing Text Categorization with Encyclopedic Knowledge," Proc. 21th Nat'l Conf. Artificial Intelligence and the 18th Innovative Applications of Artificial Intelligence Conf. (AAAI '06), July 2006.
[17] P. Wang, J. Hu, H.-J. Zeng, L. Chen, and Z. Chen, "Improving Text Classification by Using Encyclopedia Knowledge," Proc. Seventh IEEE Int'l Conf. Data Mining (ICDM '07), Oct. 2007.
[18] E. Gabrilovich and S. Markovitch, "Computing Semantic Relatedness Using Wikipedia-Based Explicit Semantic Analysis," Proc. 20th Int'l Joint Conf. Artificial Intelligence (IJCAI '07), pp. 1606-1611, Jan. 2007.
[19] X.H. Phan, M.L. Nguyen, and S. Horiguchi, "Learning to Classify Short and Sparse Text & Web with Hidden Topics from Large-Scale Data Collections," Proc. 17th Int'l Conf. World Wide Web (WWW '08), pp. 91-100, Apr. 2008.
[20] T. Hofmann, "Probabilistic Latent Semantic Analysis," Proc. 15th Conf. Uncertainty in Artificial Intelligence (UAI '99), pp. 289-296, July/Aug. 1999.
[21] D.M. Blei, A.Y. Ng, and M.I. Jordan, "Latent Dirichlet Allocation," Proc. 14st Ann. Conf. Neural Information Processing Systems (NIPS '01), pp. 601-608, Dec. 2001.
[22] P. Wang, C. Domeniconi, and J. Hu, "Using Wikipedia for Co-Clustering Based Cross-Domain Text Classification," Proc. Eighth IEEE Int'l Conf. Data Mining (ICDM '08), pp. 1085-1090, Dec. 2008.
[23] P. Wang, C. Domeniconi, and J. Hu, "Cross-Domain Text Classification Using Wikipedia," The IEEE Intelligent Informatics Bull., vol. 9, pp. 5-17, Nov. 2008.
[24] P. Wang and C. Domeniconi, "Building Semantic Kernels for Text Classification Using Wikipedia," Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining (KDD '08), pp. 713-721, Aug. 2008.
[25] S. Zelikovitz and H. Hirsh, "Improving Short-Text Classification Using Unlabeled Background Knowledge to Assess Document Similarity," Proc. 17th Int'l Conf. Machine Learning (ICML '00), pp. 1183-1190, 2000.
[26] S. Zelikovitz and H. Hirsh, "Using LSI for Text Classification in the Presence of Background Text," Proc. 10th ACM Int'l Conf. Information and Knowledge Management (CIKM '01), pp. 113-118, Nov. 2001.
[27] A. Blum and S. Chawla, "Learning from Labeled and Unlabeled Data Using Graph Mincuts," Proc. 18th Int'l Conf. Machine Learning (ICML '01), pp. 19-26, June/July 2001.
[28] A. Blum and T. Mitchell, "Combining Labeled and Unlabeled Sata with Co-Training," Proc. 11th Ann. Conf. Computational Learning Theory (COLT '98), pp. 92-100, 1998.
[29] T. Joachims, "Transductive Inference for Text Classification Using Support Vector Machines," Proc. 16th Int'l Conf. Machine Learning (ICML '99), pp. 200-209, June 1999.
[30] G. Salton, A. Wong, and C.S. Yang, "A Vector Space Model for Automatic Indexing," Comm. ACM, vol. 18, no. 11, pp. 613-620, 1975.
[31] C. Bishop, Pattern Recognition and Machine Learning. Springer-Verlag, 2006.
[32] V.W. Zheng, S.J. Pan, Q. Yang, and J.J. Pan, "Transferring Multi-Device Localization Models Using Latent Multi-Task Learning," Proc. 23rd Nat'l Conf. Artificial Intelligence (AAAI '08), pp. 1427-1432, July 2008.
[33] V.W. Zheng, E.W. Xiang, Q. Yang, and D. Shen, "Transferring Localization Models Over Time," Proc. 23rd Nat'l Conf. Artificial Intelligence (AAAI '08), pp. 1421-1426, July 2008.
[34] V.W. Zheng, D.H. Hu, and Q. Yang, "Cross-Domain Activity Recognition," Proc. 11th Int'l Conf. Ubiquitous Computing (Ubicom '09), pp. 61-70, 2009.
[35] T.L. Griffiths and M. Steyvers, "Finding Scientific Topics," Proc. Nat'l Academy of Sciences USA, vol. 101, suppl. 1, pp. 5228-5235, http://dx.doi.org/10.1073pnas.0307752101 , Apr. 2004.
[36] K. Lang, "Newsweeder: Learning to Filter Netnews," Proc. 12th Int'l Machine Learning Conf. (ICML '95), pp. 331-339, 1995.
[37] S.M. Beitzel, E.C. Jensen, O. Frieder, D.D. Lewis, A. Chowdhury, and A. Kolcz, "Improving Automatic Query Classification via Semi-Supervised Learning," Proc. Fifth IEEE Int'l Conf. Data Mining (ICDM '05), pp. 42-49, Nov. 2005.
[38] D. Shen, J.-T. Sun, Q. Yang, and Z. Chen, "Building Bridges for Web Query Classification," Proc. 29th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval (SIGIR '06), pp. 131-138, Aug. 2006.
[39] A.Z. Broder, M. Fontoura, E. Gabrilovich, A. Joshi, V. Josifovski, and T. Zhang, "Robust Classification of Rare Queries Using Web Knowledge," Proc. 30th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 231-238, July 2007.
[40] Q. Yang, Y. Chen, G.-R. Xue, W. Dai, and Y. Yu, "Heterogeneous Transfer Learning with Real-World Applications," Proc. 47th Ann. Meeting of the Assoc. for Computational Linguistics (ACL '09), Aug. 2009.
14 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool