التقييم الخالي من المرجع لديه القدرة على جعل تقييم الترجمة الآلية أكثر قابلية للتطوير بشكل كبير، مما يتيح لنا المحور بسهولة لغات أو مجالات جديدة.لقد أظهر مؤخرا أن الاحتمالات التي قدمتها نموذج كبير متعدد اللغات يمكن أن تحقق حالة من النتائج الفنية عند استخدامها كتقسيط مجاني مرجعي.نقوم بتجربة تعديلات مختلفة لهذا النموذج، وإظهار ذلك من خلال تحجيمه، يمكننا مطابقة أداء بلو.نقوم بتحليل نقاط الضعف المحتملة المختلفة للنهج، وتجد أنه قوي بشكل مدهش ومن المرجح أن تقدم أداء معقول عبر مجموعة واسعة من المجالات وصفات النظام المختلفة.
Reference-free evaluation has the potential to make machine translation evaluation substantially more scalable, allowing us to pivot easily to new languages or domains. It has been recently shown that the probabilities given by a large, multilingual model can achieve state of the art results when used as a reference-free metric. We experiment with various modifications to this model, and demonstrate that by scaling it up we can match the performance of BLEU. We analyze various potential weaknesses of the approach, and find that it is surprisingly robust and likely to offer reasonable performance across a broad spectrum of domains and different system qualities.
References used
https://aclanthology.org/
Many modern machine translation evaluation metrics like BERTScore, BLEURT, COMET, MonoTransquest or XMoverScore are based on black-box language models. Hence, it is difficult to explain why these metrics return certain scores. This year's Eval4NLP sh
Simultaneous machine translation has recently gained traction thanks to significant quality improvements and the advent of streaming applications. Simultaneous translation systems need to find a trade-off between translation quality and response time
Automatic metrics are commonly used as the exclusive tool for declaring the superiority of one machine translation system's quality over another. The community choice of automatic metric guides research directions and industrial developments by decid
Translation quality can be improved by global information from the required target sentence because the decoder can understand both past and future information. However, the model needs additional cost to produce and consider such global information.
Image captioning has conventionally relied on reference-based automatic evaluations, where machine captions are compared against captions written by humans. This is in contrast to the reference-free manner in which humans assess caption quality. In t