The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - Sept.-Oct. (2012 vol.16)
pp: 52-58
Stefan Thaler , University of Innsbruck
Elena Simperl , Karlsruhe Institute of Technology
Stephan Wölger , University of Innsbruck
ABSTRACT
Human computation can address complex computational problems by tapping into large resource pools for relatively little cost. Two prominent human-computation techniques — games with a purpose (GWAP) and microtask crowdsourcing — can help resolve semantic-technology-related tasks, including knowledge representation, ontology alignment, and semantic annotation. To evaluate which approach is better with respect to costs and benefits, the authors employ categorization challenges in Wikipedia to ultimately create a large, general-purpose ontology. They first use the OntoPronto GWAP, then replicate its problem-solving setting in Amazon Mechanical Turk, using a similar task-design structure, evaluation mechanisms, and input data.
INDEX TERMS
Games, Ontologies, Internet, Encyclopedias, Electronic publishing, Protons, Mechanical Turk, GWAP, incentives, motivators, crowdsourcing, conceptual modeling, ontology
CITATION
Stefan Thaler, Elena Simperl, Stephan Wölger, "An Experiment in Comparing Human-Computation Techniques", IEEE Internet Computing, vol.16, no. 5, pp. 52-58, Sept.-Oct. 2012, doi:10.1109/MIC.2012.67
REFERENCES
1. T. Berners-Lee, J. Hendler, and O. Lassila, "The Semantic Web," Scientific Am., vol. 284, no. 5, 2001, pp. 34–43.
2. K. Siorpaes and M. Hepp, "Games with a Purpose for the Semantic Web," IEEE Intelligent Systems, vol. 23, no. 3, 2008, pp. 50–60.
3. S. Thaler, E. Simperl, and K. Siorpaes, "SpotTheLink: A Game for Ontology Alignment," Proc. Wissenmanagement Conf., 2011, pp. 246–253.
4. L. von Ahn and L. Dabbish, "Designing Games with a Purpose," Comm. ACM, vol. 51, no. 8, 2008, pp. 58–67.
5. T.W. Malone, "What Makes Things Fun to Learn? Heuristics for Designing Instructional Computer Games," Proc. ACM SIGSMALL and SIGPC Symp. Small Systems, ACM, 1980, pp. 162–169.
6. A. Gomez-Perez, M. Fernandez-Lopez, and O. Corcho, Ontological Eng., Springer, 2004.
7. P. Buitelaar and P. Cimiano, Ontology Learning and Population: Bridging the Gap Between Text and Knowledge, IOS Press, 2008.
8. K. Siorpaes, "OntoGame: Games with a Purpose for the Semantic Web," doctoral dissertation, Semantic Technology Institute STI Innsbruck, Univ. of Innsbruck, 2009.
9. G. Little et al., "TurKit: Tools for Iterative Tasks on Mechanical Turk," Proc. ACM SIGKDD HCOMP Workshop, ACM, 2009, pp. 29–30.
10. A. Kittur, E. Chi, and B. Suh, "Crowdsourcing User Studies with Mechanical Turk," Proc. 26th Ann. SIGCHI Conf. Human Factors in Computing Systems, ACM, 2008, pp. 453–456.
11. P. Ipeirotis, F. Provost, and J. Wang, "Quality Management on Amazon Mechanical Turk," Proc. ACM SIGKDD Human Computation Workshop, ACM, 2010, pp. 64–67.
204 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool