Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userOn the Stability of System Rankings at WMT
على استقرار تصنيفات النظام في WMT
267
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
النهج الحالي لجمع الأحكام البشرية لجودة الترجمة الآلية لمهمة الترجمة الأخبار في WMT - تصنيف القطاع مع سياق المستند - هو الأحدث في سلسلة من التغييرات في بروتوكول التعليق البشري WMT.نظرا لأن البروتوكولات التوضيحية هذه تغيرت مع مرور الوقت، فقد انجرفت بعيدا عن بعض الافتراضات الإحصائية الأولية التي تدعمها، مع عواقب تسمون صحة تصنيفات نظام المهام الأخبار WMT إلى سؤال.في المحاكاة بناء على البيانات الحقيقية، نوضح أن التصنيفات يمكن أن تتأثر بوجود القيم المتطرفة (أنظمة عالية الجودة أو منخفضة الجودة)، مما أدى إلى تصنيفات ونظام مختلفة.ونحن ندرس أيضا أسئلة تكوين مهمة التوضيحية وكيف قد تؤثر سهولة ترجم أو صعوبة ترجمة المستندات المختلفة في تصنيفات النظام.نحن نقدم مناقشة طرق لتحليل هذه القضايا عند النظر في التغييرات المستقبلية في بروتوكولات التعليق التوضيحي.
The current approach to collecting human judgments of machine translation quality for the news translation task at WMT -- segment rating with document context -- is the most recent in a sequence of changes to WMT human annotation protocol. As these annotation protocols have changed over time, they have drifted away from some of the initial statistical assumptions underpinning them, with consequences that call the validity of WMT news task system rankings into question. In simulations based on real data, we show that the rankings can be influenced by the presence of outliers (high- or low-quality systems), resulting in different system rankings and clusterings. We also examine questions of annotation task composition and how ease or difficulty of translating different documents may influence system rankings. We provide discussion of ways to analyze these issues when considering future changes to annotation protocols.
References used
https://aclanthology.org/
rate research
Read More
This study aims to analyze the effect of the wind farms on frequency stability of the electrical power network, and the description of the performance of Syrian Electrical Power System with integration of
wind farms in several regions in Syria (Al-Q
uenetera – Al-Hejana - Ghabagheb) through the evaluation of frequency stability of the power system and the Critical Clearing Time (CCT).
The effect of wind farms on the frequency behavior of Syrian network and factors related will be investigated such as generation technology by replacing the power-generated source by two main types
of induction generators, changing the location of wind farms and increasing gradually the rate of wind power.
The simulation analysis will be applied on Syrian Electrical Power System 230KV – 400KV, by using program NEPLAN, which gives access to an extensive library of grid components, and relevant wind
turbine model.
Many recent approaches towards neural information retrieval mitigate their computational costs by using a multi-stage ranking pipeline. In the first stage, a number of potentially relevant candidates are retrieved using an efficient retrieval model s
uch as BM25. Although BM25 has proven decent performance as a first-stage ranker, it tends to miss relevant passages. In this context we propose CoRT, a simple neural first-stage ranking model that leverages contextual representations from pretrained language models such as BERT to complement term-based ranking functions while causing no significant delay at query time. Using the MS MARCO dataset, we show that CoRT significantly increases the candidate recall by complementing BM25 with missing candidates. Consequently, we find subsequent re-rankers achieve superior results with less candidates. We further demonstrate that passage retrieval using CoRT can be realized with surprisingly low latencies.
The machine translation efficiency task challenges participants to make their systems faster and smaller with minimal impact on translation quality. How much quality to sacrifice for efficiency depends upon the application, so participants were encou
raged to make multiple submissions covering the space of trade-offs. In total, there were 53 submissions by 4 teams. There were GPU, single-core CPU, and multi-core CPU hardware tracks as well as batched throughput or single-sentence latency conditions. Submissions showed hundreds of millions of words can be translated for a dollar, average latency is 5--17 ms, and models fit in 7.5--150 MB.
Language domains that require very careful use of terminology are abundant and reflect a significant part of the translation industry. In this work we introduce a benchmark for evaluating the quality and consistency of terminology translation, focusi
ng on the medical (and COVID-19 specifically) domain for five language pairs: English to French, Chinese, Russian, and Korean, as well as Czech to German. We report the descriptions and results of the participating systems, commenting on the need for further research efforts towards both more adequate handling of terminologies as well as towards a proper formulation and evaluation of the task.
We present our development of the multilingual machine translation system for the large-scale multilingual machine translation task at WMT 2021. Starting form the provided baseline system, we investigated several techniques to improve the translation
quality on the target subset of languages. We were able to significantly improve the translation quality by adapting the system towards the target subset of languages and by generating synthetic data using the initial model. Techniques successfully applied in zero-shot multilingual machine translation (e.g. similarity regularizer) only had a minor effect on the final translation performance.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
