Type

Conference Proceedings

Authors

Wessel Kraaij
Paul Over
Alan F. Smeaton

Subjects

Computer Science

Topics
benchmarking information retrieval video retrieval video information retrieval low level features automatic detection digital video

Evaluation campaigns and TRECVid (2006)

Abstract The TREC Video Retrieval Evaluation (TRECVid) is an international benchmarking activity to encourage research in video information retrieval by providing a large test collection, uniform scoring procedures, and a forum for organizations interested in comparing their results. TRECVid completed its fifth annual cycle at the end of 2005 and in 2006 TRECVid will involve almost 70 research organizations, universities and other consortia. Throughout its existence, TRECVid has benchmarked both interactive and automatic/manual searching for shots from within a video corpus, automatic detection of a variety of semantic and low-level video features, shot boundary detection and the detection of story boundaries in broadcast TV news. This paper will give an introduction to information retrieval (IR) evaluation from both a user and a system perspective, highlighting that system evaluation is by far the most prevalent type of evaluation carried out. We also include a summary of TRECVid as an example of a system evaluation benchmarking campaign and this allows us to discuss whether such campaigns are a good thing or a bad thing. There are arguments for and against these campaigns and we present some of them in the paper concluding that on balance they have had a very positive impact on research progress.
Collections Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> Subject = Computer Science
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: Centre for Digital Video Processing (CDVP)
Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: Adaptive Information Cluster (AIC)
Ireland -> Dublin City University -> Subject = Computer Science: Information retrieval
Ireland -> Dublin City University -> Subject = Computer Science: Digital video
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres

Full list of authors on original publication

Wessel Kraaij, Paul Over, Alan F. Smeaton

Experts in our system

1
Alan F. Smeaton
Dublin City University
Total Publications: 450