Type

Conference Proceedings

Authors

Alan F. Smeaton
Adam Bermingham

Subjects

Linguistics

Topics
information retrieval agreement human sentiment analysis individual computational linguistics machine learning large scale

A study of inter-annotator agreement for opinion retrieval (2009)

Abstract Evaluation of sentiment analysis, like large-scale IR evalu- ation, relies on the accuracy of human assessors to create judgments. Subjectivity in judgments is a problem for rel- evance assessment and even more so in the case of senti- ment annotations. In this study we examine the degree to which assessors agree upon sentence-level sentiment anno- tation. We show that inter-assessor agreement is not con- tingent on document length or frequency of sentiment but correlates positively with automated opinion retrieval per- formance. We also examine the individual annotation cate- gories to determine which categories pose most di┬▒culty for annotators.
Collections Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> Subject = Computer Science: Computational linguistics
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres
Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Computing
Ireland -> Dublin City University -> Subject = Computer Science
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: CLARITY: The Centre for Sensor Web Technologies
Ireland -> Dublin City University -> Subject = Computer Science: Information retrieval
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing
Ireland -> Dublin City University -> Subject = Computer Science: Machine learning

Full list of authors on original publication

Alan F. Smeaton, Adam Bermingham

Experts in our system

1
Alan F. Smeaton
Dublin City University
Total Publications: 475
 
2
Adam Bermingham
Dublin City University
Total Publications: 15