Type

Conference Proceedings

Authors

Andy Way
Sheila Castilho
Pintu Lohar
Evgeny Matusov
Daniel Stein
Iacer Calixto

Subjects

Language

Topics
images training data state of the art approaches general statistical machine translation machine translating multi modal

Using images to improve machine-translating E-commerce product listings (2017)

Abstract In this paper we study the impact of using images to machine-translate user-generated ecommerce product listings. We study how a multi-modal Neural Machine Translation (NMT) model compares to two text-only approaches: a conventional state-of-the-art attentional NMT and a Statistical Machine Translation (SMT) model. User-generated product listings often do not constitute grammatical or well-formed sentences. More often than not, they consist of the juxtaposition of short phrases or keywords. We train our models end-to-end as well as use text-only and multimodal NMT models for re-ranking n-best lists generated by an SMT model. We qualitatively evaluate our user-generated training data also analyse how adding synthetic data impacts the results. We evaluate our models quantitatively using BLEU and TER and find that (i) additional synthetic data has a general positive impact on text-only and multi-modal NMT models, and that (ii) using a multi-modal NMT model for re-ranking n-best lists improves TER significantly across different nbest list sizes.
Collections Ireland -> Dublin City University -> Publication Type = Conference or Workshop Item
Ireland -> Dublin City University -> DCU Faculties and Centres = DCU Faculties and Schools: Faculty of Engineering and Computing: School of Computing
Ireland -> Dublin City University -> DCU Faculties and Centres = Research Initiatives and Centres: ADAPT
Ireland -> Dublin City University -> Status = Published
Ireland -> Dublin City University -> Subject = Computer Science: Machine translating

Full list of authors on original publication

Andy Way, Sheila Castilho, Pintu Lohar, Evgeny Matusov, Daniel Stein, Iacer Calixto

Experts in our system

1
Andy Way
Dublin City University
Total Publications: 229
 
2
Sheila Castilho
Dublin City University
Total Publications: 16
 
3
Pintu Lohar
Dublin City University
Total Publications: 10
 
4
Daniel Stein
Dublin City University
Total Publications: 6
 
5
Iacer Calixto
Dublin City University
Total Publications: 6