早期退出与即时置信度翻译质量评估

摘要

质量评估在机器翻译中无处不在，无论是用于评估还是生成环节。然而，质量评估模型往往不透明且计算成本高昂，这使得它们难以融入大规模处理流程。本研究致力于解决两个相互关联的挑战：(1) 降低大规模质量评估的成本，(2) 开发一种低成本的质量评估不确定性估计方法。针对后者，我们提出了Instant Confidence COMET，这是一种具备不确定性感知能力的质量评估模型，它以极低的成本达到了以往方法的性能水平。我们进一步将其扩展为Early-Exit COMET，这种质量评估模型能够在模型早期层级就计算出质量分数及相应的置信度，从而允许我们提前终止计算，降低评估成本。此外，我们还将该模型应用于机器翻译的重排序任务中。通过将Early-Exit COMET与上置信区间多臂赌博机算法结合，我们能够从大量候选翻译中找出最佳选项，而无需对所有候选执行完整的评估模型。无论是评估还是重排序场景，我们的方法均将所需计算量减少了50%，同时性能损失微乎其微。

English

Quality estimation is omnipresent in machine translation, for both evaluation and generation. Unfortunately, quality estimation models are often opaque and computationally expensive, making them impractical to be part of large-scale pipelines. In this work, we tackle two connected challenges: (1) reducing the cost of quality estimation at scale, and (2) developing an inexpensive uncertainty estimation method for quality estimation. To address the latter, we introduce Instant Confidence COMET, an uncertainty-aware quality estimation model that matches the performance of previous approaches at a fraction of their costs. We extend this to Early-Exit COMET, a quality estimation model that can compute quality scores and associated confidences already at early model layers, allowing us to early-exit computations and reduce evaluation costs. We also apply our model to machine translation reranking. We combine Early-Exit COMET with an upper confidence bound bandit algorithm to find the best candidate from a large pool without having to run the full evaluation model on all candidates. In both cases (evaluation and reranking) our methods reduce the required compute by 50% with very little degradation in performance.

早期退出与即时置信度翻译质量评估

Early-Exit and Instant Confidence Translation Quality Estimation

摘要

Summary

Support