Type

Conference Proceedings

Authors

Roeland Ordelman
Martha Larson
Gareth J. F. Jones
Maria Eskevich

Subjects

Computer Science

Topics
crowdsourcing data collection information retrieval speech collection creation multimedia systems speech search speech retrieval rich speech retrieval

Creating a data collection for evaluating rich speech retrieval (2012)

Abstract We describe the development of a test collection for the investigation of speech retrieval beyond identification of relevant content. This collection focuses on satisfying user information needs for queries associated with specific types of speech acts. The collection is based on an archive of the Internet video from Internet video sharing platform (blip.tv), and was provided by the MediaEval benchmarking initiative. A crowdsourcing approach was used to identify segments in the video data which contain speech acts, to create a description of the video containing the act and to generate search queries designed to refind this speech act. We describe and reflect on our experiences with crowdsourcing this test collection using the Amazon Mechanical Turk platform. We highlight the challenges of constructing this dataset, including the selection of the data source, design of the crowdsouring task and the specification of queries and relevant items.
Collections Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: Centre for Digital Video Processing (CDVP)
Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> Subject = Computer Science: Multimedia systems
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres
Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Computing
Ireland -> Dublin City University -> Subject = Computer Science
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: Centre for Next Generation Localisation (CNGL)
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools
Ireland -> Dublin City University -> Subject = Computer Science: Information retrieval
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing

Full list of authors on original publication

Roeland Ordelman, Martha Larson, Gareth J. F. Jones, Maria Eskevich

Experts in our system

1
Roeland Ordelman
Dublin City University
Total Publications: 11
 
2
Gareth J. F. Jones
Dublin City University
Total Publications: 265
 
3
Maria Eskevich
Dublin City University
Total Publications: 19