Document type
Journal articles
Document subtype
Full paper
Title
Onception: Active Learning with Expert Advice for Real World Machine Translation
Participants in the publication
Vânia Mendonça (Author)
Ricardo Rei (Author)
INSTITUTO SUPERIOR TÉCNICO
Luísa Coheur (Author)
INSTITUTO SUPERIOR TÉCNICO
Alberto Sardinha (Author)
INSTITUTO SUPERIOR TÉCNICO
Summary
Active learning can play an important role in low-resource settings (i.e., where annotated data is scarce), by selecting which instances may be more worthy to annotate. Most active learning approaches for Machine Translation assume the existence of a pool of sentences in a source language, and rely on human annotators to provide translations or post-edits, which can still be costly. In this article, we apply active learning to a real-world human-in-the-loop scenario in which we assume that: (1) the source sentences may not be readily available, but instead arrive in a stream; (2) the automatic translations receive feedback in the form of a rating, instead of a correct/edited translation, since the human-in-the-loop might be a user looking for a translation, but not be able to provide one. To tackle the challenge of deciding whether each incoming pair source–translations is worthy to query for human feedback, we resort to a number of stream-based active learning query strategies. Moreover, because we do not know in advance which query strategy will be the most adequate for a certain language pair and set of Machine Translation models, we propose to dynamically combine multiple strategies using prediction with expert advice. Our experiments on different language pairs and feedback settings show that using active learning allows us to converge on the best Machine Translation systems with fewer human interactions. Furthermore, combining multiple strategies using prediction with expert advice outperforms several individual active learning strategies with even fewer interactions, particularly in partial feedback settings.
Date of Publication
2023-01-13
Where published
Computational Linguistics
Publication Identifiers
ISSN - 0891-2017
Publisher
MIT Press
Number of pages
48
Starting page
1
Last page
48
Document Identifiers
DOI -
https://doi.org/10.1162/coli_a_00473
URL -
http://dx.doi.org/10.1162/coli_a_00473
Rankings
SCIMAGO Q1 (2021) - 103 - Computer Science Applications