Type

Conference Proceedings

Authors

Maria Gialama
Antonio Valerio Miceli Barone
Andy Way
Pintu Lohar
Panayota Georgakopoulou
Vilelmini Sosoni
Rico Sennrich
Federico Gaspari
Joss Moorkens
Sheila Castilho

Subjects

Linguistics

Topics
post editing effort comparative human language pairs professional translation quality phrase based statistical machine translation machine translating

A comparative quality evaluation of PBSMT and NMT using professional translators (2017)

Abstract This paper reports on a comparative evaluation of phrase-based statistical machine translation (PBSMT) and neural machine translation (NMT) for four language pairs, using the PET interface to compare educational domain output from both systems using a variety of metrics, including automatic evaluation as well as human rankings of adequacy and fluency, error-type markup, and post-editing (technical and temporal) effort, performed by professional translators. Our results show a preference for NMT in side-by-side ranking for all language pairs, texts, and segment lengths. In addition, perceived fluency is improved and annotated errors are fewer in the NMT output. Results are mixed for perceived adequacy and for errors of omission, addition, and mistranslation. Despite far fewer segments requiring post-editing, document-level post-editing performance was not found to have significantly improved in NMT compared to PBSMT. This evaluation was conducted as part of the TraMOOC project, which aims to create a replicable semi-automated methodology for high-quality machine translation of educational data.
Collections Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Computing
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: ADAPT
Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> Subject = Computer Science: Machine translating

Full list of authors on original publication

Maria Gialama, Antonio Valerio Miceli Barone, Andy Way, Pintu Lohar, Panayota Georgakopoulou, Vilelmini Sosoni, Rico Sennrich, Federico Gaspari, Joss Moorkens, Sheila Castilho

Experts in our system

1
Andy Way
Dublin City University
Total Publications: 229
 
2
Pintu Lohar
Dublin City University
Total Publications: 10
 
3
Panayota Georgakopoulou
Dublin City University
Total Publications: 4
 
4
Vilelmini Sosoni
Dublin City University
Total Publications: 4
 
5
Rico Sennrich
Dublin City University
Total Publications: 3
 
6
Federico Gaspari
Dublin City University
Total Publications: 9
 
7
Joss Moorkens
Dublin City University
Total Publications: 17
 
8
Sheila Castilho
Dublin City University
Total Publications: 16