This Article 
 Bibliographic References 
 Add to: 
Evaluation Challenges and Directions for Information-Seeking Support Systems
March 2009 (vol. 42 no. 3)
pp. 60-66
Diane Kelly, University of North Carolina at Chapel Hill
Susan Dumais, Microsoft Research
ISSSs provide an exciting opportunity to extend previous information-seeking and interactive information retrieval evaluation models and create a research community that embraces diverse methods and broader participation.

1. C. Cleverdon, "The Cranfield Tests on Index Language Devices," Readings in Information Retrieval, K.S. Jones, and P. Willett eds., Morgan Kaufmann, 1997, pp. 47-59.
2. E.M. Voorhees, and D.K. Harman eds., , TREC: Experiment and Evaluation in Information Retrieval, MIT Press, 2005.
3. I. Ruthven, "Interactive Information Retrieval," Ann. Rev. Information Science and Technology, vol. 42, 2008, pp. 43-91.
4. S.T. Dumais and N.J. Belkin, "The TREC Interactive Tracks: Putting the User into Search," TREC: Experiment and Evaluation in Information Retrieval, E.M. Voorhees, and D.K. Harman eds., MIT Press, pp. 123-153.
5. C. Grimes, D. Tang, and D.M. Russell, "Query Logs Alone Are Not Enough," Proc. Workshop on Query Log Analysis at the 16th Int'l World Wide Web Conf., ACM Press, 2007;
6. K. Järvelin et al., "Discounted Cumulated Gain-Based Evaluation of Multiple-Query IR Sessions," Advances in Information Retrieval, LNCS 4956, Springer, 2008, pp. 4-15.
7. M.J. Bates, "The Design of Browsing and Berrypicking Techniques for the Online Search Interface," Online Rev., Oct. 1989, pp. 407-424.
8. C.C. Kuhlthau, Seeking Meaning: A Process Approach to Library and Information Services, 2nd ed., Libraries Unlimited, 2003.
9. T. Saracevic, "Relevance: A Review of the Literature and a Framework for Thinking on the Notion in Information Science. Part II: Nature and Manifestations of Relevance," J. Am. Soc. Information Science and Technology, Nov. 2007, pp. 1915-1933.
10. O. Alonso, D.E. Rose, and B. Stewart, "Crowdsourcing for Relevance Evaluation," ACM SIGIR Forum, Dec. 2008, pp. 9-15.

Index Terms:
information retrieval, information-seeking support systems, ISSSs, evaluation design
Diane Kelly, Susan Dumais, Jan O. Pedersen, "Evaluation Challenges and Directions for Information-Seeking Support Systems," Computer, vol. 42, no. 3, pp. 60-66, March 2009, doi:10.1109/MC.2009.82
Usage of this product signifies your acceptance of the Terms of Use.