Type

Journal Article

Authors

Guido Zuccon
Joao Palotti
Mihai Lupu
Johannes Leveling
Liadh Kelly
Gareth J.F. Jones
Lorraine Goeuriot

Subjects

Computer Science

Topics
ir test collections ad hoc quality characteristics medical information retrieval health query expansion comparative

An Analysis of Evaluation Campaigns in ad-hoc Medical Information Retrieval: CLEF eHealth 2013 and 2014 (2018)

Abstract Since its inception in 2013, one of the key contributions of the CLEF eHealth evaluation campaign has been the organization of an ad-hoc information retrieval (IR) benchmarking task. This IR task evaluates systems intended to support laypeople searching for and understanding health infor- mation. Each year the task provides registered participants with standard IR test collections consisting of a document collection and topic set. Partici- pants then return retrieval results obtained by their IR systems for each query, which are assessed using a pooling procedure. In this article we focus on CLEF eHealth 2013 and 2014's retrieval task, which saw topics created based on pa- tients' information needs associated with their medical discharge summaries. We overview the task and datasets created, and the results obtained by par- ticipating teams over these two years. We then provide a detailed comparative analysis of the results, and conduct an evaluation of the datasets in the light of these results. This two-fold study of the evaluation campaign teaches us about technical aspects of medical IR, such as the e´┐Żectiveness of query expansion; the quality and characteristics of CLEF eHealth IR datasets, such as their reliability; and how to run an IR evaluation campaign in the medical domain.
Collections Ireland -> Maynooth University -> Type = Article
Ireland -> Maynooth University -> Status = Published
Ireland -> Maynooth University -> Academic Unit = Faculty of Science and Engineering: Computer Science

Full list of authors on original publication

Guido Zuccon, Joao Palotti, Mihai Lupu, Johannes Leveling, Liadh Kelly, Gareth J.F. Jones, Lorraine Goeuriot

Experts in our system

1
Guido Zuccon
Dublin City University
Total Publications: 10
 
2
Joao Palotti
Dublin City University
Total Publications: 5
 
3
Johannes Leveling
Dublin City University
Total Publications: 66
 
4
Liadh Kelly
Dublin City University
Total Publications: 51
 
5
Gareth J. F. Jones
Dublin City University
Total Publications: 297
 
6
Lorraine Goeuriot
Dublin City University
Total Publications: 24