Type

Conference Proceedings

Authors

Andy Way
Sheila Castilho
Evgeny Matusov
Daniel Stein
Iacer Calixto

Subjects

Linguistics

Topics
machine translating multi modal phrase based statistical machine translation state of the art evaluation study titles machine translation evaluation human

Human evaluation of multi-modal neural machine translation: a case study on E-commerce listing titles (2017)

Abstract In this paper, we study how humans perceive the use of images as an additional knowledge source to machine-translate usergenerated product listings in an e-commerce company. We conduct a human evaluation where we assess how a multi-modal neural machine translation (NMT) model compares to two text-only approaches: a conventional state-of-the-art attention-based NMT and a phrase-based statistical machine translation (PBSMT) model. We evaluate translations obtained with different systems and also discuss the data set of user-generated product listings, which in our case comprises both product listings and associated images. We found that humans preferred translations obtained with a PBSMT system to both text-only and multi-modal NMT over 56% of the time. Nonetheless, human evaluators ranked translations from a multi-modal NMT model as better than those of a text-only NMT over 88% of the time, which suggests that images do help NMT in this use-case.
Collections Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Computing
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: ADAPT
Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> Subject = Computer Science: Machine translating

Full list of authors on original publication

Andy Way, Sheila Castilho, Evgeny Matusov, Daniel Stein, Iacer Calixto

Experts in our system

1
Andy Way
Dublin City University
Total Publications: 229
 
2
Sheila Castilho
Dublin City University
Total Publications: 16
 
3
Daniel Stein
Dublin City University
Total Publications: 6
 
4
Iacer Calixto
Dublin City University
Total Publications: 6