في هذه الورقة، نقترح نموذجا مقرا له عناية سياقية مع تدريبات دقيقة على مرحلتين باستخدام روبرتا.أولا، نقوم بإجراء النغمة الجميلة في المرحلة الأولى على Corpus مع روبرتا، بحيث يمكن للنموذج أن يتعلم بعض المعرفة المسبقة المجال.ثم نحصل على التضمين السياقي بكلمات السياق بناء على التضمين على مستوى الرمز المميز مع النموذج الدقيق.ونحن نستخدم KFOFT التحقق من الصحة للحصول على نماذج K وفرقة لهم للحصول على النتيجة النهائية.أخيرا، نحن نحصل على المركز الثاني في مرحلة التقييم النهائي من المهمة الفرعية 2 مع ارتباط بيرسون ب 0.8575.
In this paper we propose a contextual attention based model with two-stage fine-tune training using RoBERTa. First, we perform the first-stage fine-tune on corpus with RoBERTa, so that the model can learn some prior domain knowledge. Then we get the contextual embedding of context words based on the token-level embedding with the fine-tuned model. And we use Kfold cross-validation to get K models and ensemble them to get the final result. Finally, we attain the 2nd place in the final evaluation phase of sub-task 2 with pearson correlation of 0.8575.
References used
https://aclanthology.org/
Lexical complexity prediction (LCP) conveys the anticipation of the complexity level of a token or a set of tokens in a sentence. It plays a vital role in the improvement of various NLP tasks including lexical simplification, translations, and text g
This paper describes a system submitted by team BigGreen to LCP 2021 for predicting the lexical complexity of English words in a given context. We assemble a feature engineering-based model with a deep neural network model founded on BERT. While BERT
We propose an ensemble model for predicting the lexical complexity of words and multiword expressions (MWEs). The model receives as input a sentence with a target word or MWE and outputs its complexity score. Given that a key challenge with this task
This paper describes team LCP-RIT's submission to the SemEval-2021 Task 1: Lexical Complexity Prediction (LCP). The task organizers provided participants with an augmented version of CompLex (Shardlow et al., 2020), an English multi-domain dataset in
Lexical complexity plays an important role in reading comprehension. lexical complexity prediction (LCP) can not only be used as a part of Lexical Simplification systems, but also as a stand-alone application to help people better reading. This paper