Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAre References Really Needed? Unbabel-IST 2021 Submission for the Metrics Shared Task
هل هناك حاجة مراجع حقا؟GREXABEL-IST 2021 تقديم المهمة المشتركة للمقاييس
360
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
في هذه الورقة، نقدم مساهمة مشتركة من المهمة المشتركة ومقاييس WMT 2021.مع تركيز هذا العام على متري الجودة متعددة الأبعاد (MQM) باعتباره التقييم البشري الحقيقة الأرضية، كان هدفنا هو توجيه المذنب نحو الارتباطات الأعلى مع MQM.نحن نقوم بذلك عن طريق التدريب المسبق أولا على التقييمات المباشرة ثم ضبط نتائج MQM تطبيع Z.في تجاربنا، نعرض أيضا أن نماذج المذنب المجانية المرجعية أصبحت تنافسية للنماذج القائمة على المراجع، حتى تتفوق على أفضل نموذج مذنب من عام 2020 في بيانات تطوير هذا العام.بالإضافة إلى ذلك، نقدم Cometinho، وهو نموذج مذنب خفيف الوزن هو 19x أسرع في وحدة المعالجة المركزية من النموذج الأصلي، مع تحقيق الارتباطات الحديثة أيضا مع MQM.أخيرا، في QE كمسار متري، شاركنا أيضا مع نموذج QE المدرب باستخدام إطار OpenKiwi الاستفادة من درجات MQM وشروح مستوى الكلمة.
In this paper, we present the joint contribution of Unbabel and IST to the WMT 2021 Metrics Shared Task. With this year's focus on Multidimensional Quality Metric (MQM) as the ground-truth human assessment, our aim was to steer COMET towards higher correlations with MQM. We do so by first pre-training on Direct Assessments and then fine-tuning on z-normalized MQM scores. In our experiments we also show that reference-free COMET models are becoming competitive with reference-based models, even outperforming the best COMET model from 2020 on this year's development data. Additionally, we present COMETinho, a lightweight COMET model that is 19x faster on CPU than the original model, while also achieving state-of-the-art correlations with MQM. Finally, in the QE as a metric'' track, we also participated with a QE model trained using the OpenKiwi framework leveraging MQM scores and word-level annotations.
References used
https://aclanthology.org/
rate research
Read More
We present the joint contribution of IST and Unbabel to the WMT 2021 Shared Task on Quality Estimation. Our team participated on two tasks: Direct Assessment and Post-Editing Effort, encompassing a total of 35 submissions. For all submissions, our ef
forts focused on training multilingual models on top of OpenKiwi predictor-estimator architecture, using pre-trained multilingual encoders combined with adapters. We further experiment with and uncertainty-related objectives and features as well as training on out-of-domain direct assessment data.
This paper describes Papago submission to the WMT 2021 Quality Estimation Task 1: Sentence-level Direct Assessment. Our multilingual Quality Estimation system explores the combination of Pretrained Language Models and Multi-task Learning architecture
s. We propose an iterative training pipeline based on pretraining with large amounts of in-domain synthetic data and finetuning with gold (labeled) data. We then compress our system via knowledge distillation in order to reduce parameters yet maintain strong performance. Our submitted multilingual systems perform competitively in multilingual and all 11 individual language pair settings including zero-shot.
In this paper, we describe our submission to the WMT 2021 Metrics Shared Task. We use the automatically-generated questions and answers to evaluate the quality of Machine Translation (MT) systems. Our submission builds upon the recently proposed MTEQ
A framework. Experiments on WMT20 evaluation datasets show that at the system-level the MTEQA metric achieves performance comparable with other state-of-the-art solutions, while considering only a certain amount of information from the whole translation.
This paper presents the ROCLING 2021 shared task on dimensional sentiment analysis for educational texts which seeks to identify a real-value sentiment score of self-evaluation comments written by Chinese students in the both valence and arousal dime
nsions. Valence represents the degree of pleasant and unpleasant (or positive and negative) feelings, and arousal represents the degree of excitement and calm. Of the 7 teams registered for this shared task for two-dimensional sentiment analysis, 6 submitted results. We expected that this evaluation campaign could produce more advanced dimensional sentiment analysis techniques for the educational domain. All data sets with gold standards and scoring script are made publicly available to researchers.
This paper presents the JHU-Microsoft joint submission for WMT 2021 quality estimation shared task. We only participate in Task 2 (post-editing effort estimation) of the shared task, focusing on the target-side word-level quality estimation. The tech
niques we experimented with include Levenshtein Transformer training and data augmentation with a combination of forward, backward, round-trip translation, and pseudo post-editing of the MT output. We demonstrate the competitiveness of our system compared to the widely adopted OpenKiwi-XLM baseline. Our system is also the top-ranking system on the MT MCC metric for the English-German language pair.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
