This Article 
 Bibliographic References 
 Add to: 
Learning Users' Interests by Quality Classification in Market-Based Recommender Systems
December 2005 (vol. 17 no. 12)
pp. 1678-1688
Recommender systems are widely used to cope with the problem of information overload and, to date, many recommendation methods have been developed. However, no one technique is best for all users in all situations. To combat this, we have previously developed a market-based recommender system that allows multiple agents (each representing a different recommendation method or system) to compete with one another to present their best recommendations to the user. In our system, the marketplace encourages good recommendations by rewarding the corresponding agents who supplied them according to the users' ratings of their suggestions. Moreover, we have theoretically shown how our system incites the agents to bid in a manner that ensures only the best recommendations are presented. To do this effectively in practice, however, each agent needs to be able to classify its recommendations into different internal quality levels, learn the users' interests for these different levels, and then adapt its bidding behavior for the various levels accordingly. To this end, in this paper, we develop a reinforcement learning and Boltzmann exploration strategy that the recommending agents can exploit for these tasks. We then demonstrate that this strategy does indeed help the agents to effectively obtain information about the users' interests which, in turn, speeds up the market convergence and enables the system to rapidly highlight the best recommendations.

[1] P. Resnick and H.R. Varian, “Recommender Systems,” Comm. ACM, vol. 40, no. 3, pp. 56-58, 1997.
[2] U. Shardanand and P. Maes, “Social Information Filtering: Algorithms for Automating ‘Word of Mouth,’” Proc. Conf. Human Factors in Computing Systems, CHI'95, pp. 210-217, 1995.
[3] W. Hill, L. Stead, M. Rosenstein, and G. Furnas, “Recommending and Evaluating Choices in a Virtual Community of Use,” Proc. Conf. Human Factors in Computing Systems, CHI'95, pp. 194-201, 1995.
[4] L. Terveen and W. Hill, “Human-Computer Collaboration in Recommended Systems,” Human-Computer Interaction in the New Millennium, chapter 22, J. Carroll, ed. Addison Wesley, 2001.
[5] J. Herlocker, J. Konstan, L. Terveen, and J. Riedl, “Evaluating Collaborative Filtering Recommender Systems,” ACM Trans. Information Systems, vol. 22, no. 1, pp. 5-53, 2004.
[6] Y.Z. Wei, L. Moreau, and N.R. Jennings, “Recommender Systems: A Market-Based Design,” Proc. Int'l Conf. Autonomous Agents and Multi Agent Systems (AAMAS03), pp. 600-607, 2003.
[7] Y.Z. Wei, L. Moreau, and N.R. Jennings, “Market-Based Recommendations: Design, Simulation, and Evaluation,” Agent-Oriented Information Systems, LNAI 3030, P. Giorgini, B. Henderson-Sellers, and M. Winikoff, eds., pp. 63-78, Springer-Verlag, 2004.
[8] S. Zhang, Q. Yang, and C. Zhang, “Data Preparation for Data Mining,” Applied Artificial Intelligence, vol. 17, pp. 375-381, 2003.
[9] S. Zhang, C. Zhang, and Q. Yang, “Information Enhancement for Data Mining,” IEEE Intelligent Systems, pp. 12-13, Mar./Apr. 2004.
[10] Y.Z. Wei, L. Moreau, and N.R. Jennings, “Market-Based Recommender Systems: Learning Users' Interests by Quality Classification,” Proc. Sixth Int'l Bi-Conf. Workshop Agent-Oriented Information Systems (AOIS-2004), pp. 119-133, 2004.
[11] T. Mitchell, Machine Learning. McGraw Hill, 1997.
[12] L.P. Kaelbling, M.L. Littman, and A.W. Moore, “Reinforcement Learning: A Survey,” J. Artificial Intelligence Research, vol. 4, pp. 237-285, 1996.
[13] E.L. Thorndike, “Animal Intelligence: An Experimental Study of the Associative Processes in Animals,” Psychological Rev., Monograph Supplements, no. 8, New York: MacMillan, 1898.
[14] J.A. Konstan, B.N. Miller, D. Maltz, J.L. Herlocker, L.R. Gordon, and J. Riedl, “Grouplens: Applying Collaborative Filtering to Usenet News,” Comm. ACM, vol. 40, no. 3, pp. 77-87, 1997.
[15] S.M. Bohte, E. Gerding, and H. La Poutre, “Market-Based Recommendation: Agents that Compete for Consumer Attention,” ACM Trans. Internet Technology, vol. 4, no. 4, pp. 420-448, 2004.
[16] R.J. Mooney and L. Roy, “Content-Based Book Recommending Using Learning for Text Categorization,” Proc. Fifth ACM Conf. Digital Libraries, pp. 195-204, 2000.
[17] M. Montaner, B. Lopez, and J.L. Dela, “A Taxonomy of Recommender Agents on the Internet,” Artificial Intelligence Rev., vol. 19, pp. 285-330, 2003.
[18] B.M. Sarwar, J.A. Konstan, A. Borchers, J. Herlocker, B. Miller, and J. Riedl, “Using Filtering Agents to Improve Prediction Quality in the Grouplens Research Collaborative Filtering System,” Proc. 1998 ACM Conf. Computer Supported Cooperative Work, pp. 345-354, 1998.
[19] J.L. Herlocker, J.A. Konstan, and J. Riedl, “Explaining Collaborative Filtering Recommendations,” Proc. ACM Conf. Computer Supported Cooperative Work, pp. 241-250, 2000.
[20] M. Pazzani, “A Framework for Collaborative, Content-Based and Demographic Filtering,” Artificial Intelligence Rev., vol. 13, nos. 5-6, pp. 393-408, 1999.
[21] A. Popescul, L.H. Ungar, D.M. Pennock, and S. Lawrence, “Probabilistic Models for Unified Collaborative and Content-Based Recommendation in Sparse-Data Environments,” Proc. 17th Conf. Uncertainty in Artificial Intelligence (UAI-2001), pp. 437-444, 2001.
[22] D.A. Berry and B. Fristedt, Bandit Problems: Sequential Allocation of Experiments. London: Chapman and Hall, 1985.
[23] J.C. Gittins, Multi-Armed Bandit Allocation Indices. Wiley, 1989.
[24] S.B. Thrun, “The Role of Exploration in Learning Control,” Handbook of Intelligent Control, D.A. White and D.A. Sofge, eds., New York: Van Nostrand, 1992.
[25] L.P. Kaelbling, Learning in Embedded Systems. Cambridge, Mass.: MIT Press, 1993.

Index Terms:
Index Terms- Information filtering, machine learning, recommender systems, markets.
Yan Zheng Wei, Luc Moreau, Nicholas R. Jennings, "Learning Users' Interests by Quality Classification in Market-Based Recommender Systems," IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 12, pp. 1678-1688, Dec. 2005, doi:10.1109/TKDE.2005.200
Usage of this product signifies your acceptance of the Terms of Use.