Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCode to Comment Translation: A Comparative Study on Model Effectiveness \& Errors
رمز التعليق الترجمة: دراسة مقارنة حول فعالية النموذج \ أخطاء
352
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تلخيص شفرة المصدر الآلي هي موضوع أبحاث هندسة برامج شهيرة حيث يتم استخدام نماذج الترجمة الآلية لترجمة مقتطفات التعليمات البرمجية إلى أوصاف اللغة الطبيعية ذات الصلة. تتم معظم تقييمات هذه النماذج باستخدام مقاييس تلقائية قائمة على المرجع. ومع ذلك، بالنظر إلى الفجوة الدلالية الكبيرة نسبيا بين لغات البرمجة واللغة الطبيعية، فإننا نقول أن هذا الخط من الأبحاث سيستفيد من التحقيق النوعي في أوضاع الخطأ المختلفة لنماذج الحالة الحالية. لذلك، في هذا العمل، نقوم بإجراء مقارنة كمية ونوعية من ثلاثة نماذج تلخيص شفرة المصدر المقترحة مؤخرا. في تقييمنا الكمي، قارن النماذج المستندة إلى مقاييس الترجمة من BLEU-4 و Noteor و Rouge-L، وفي تقييمنا النوعي، نقوم بإجراء ترميز مفتوح يدوي لأخطاء الأكثر شيوعا التي ارتكبتها النماذج مقارنة مع تسميات الحقيقة الأرضية. يكشف تحقيقنا عن رؤى جديدة في العلاقة بين الأداء المستند إلى الأداء المتري وأخطاء التنبؤ بالنماذج التي تستند إلى تصنيف خطأ يمكن استخدامها لدفع جهود البحث في المستقبل.
Automated source code summarization is a popular software engineering research topic wherein machine translation models are employed to translate'' code snippets into relevant natural language descriptions. Most evaluations of such models are conducted using automatic reference-based metrics. However, given the relatively large semantic gap between programming languages and natural language, we argue that this line of research would benefit from a qualitative investigation into the various error modes of current state-of-the-art models. Therefore, in this work, we perform both a quantitative and qualitative comparison of three recently proposed source code summarization models. In our quantitative evaluation, we compare the models based on the smoothed BLEU-4, METEOR, and ROUGE-L machine translation metrics, and in our qualitative evaluation, we perform a manual open-coding of the most common errors committed by the models when compared to ground truth captions. Our investigation reveals new insights into the relationship between metric-based performance and model prediction errors grounded in an error taxonomy that can be used to drive future research efforts.
References used
https://aclanthology.org/
rate research
Read More
Electronic arbitration and its electronic decisions are
considered to be the most modern means of resolving disputes
arising from electronic business transactions, and since the
electronic arbitration decision issued by the electronic arbitral
tr
ibunal is done electronically both in terms of writing or signing.
The majority of the legislations of the various countries of the
world lack it and the obligation of the losing party in this decision
to implement its obligation by judicial means or by alternative
means has not This is the subject of our research, which we have
been forced to distribute to two sections, the first of which dealt
with the nature of electronic arbitration decisions and our second
release to ensure the effectiveness of the electronic arbitration
decision. We finished the research by concluding the subject.
Frame-based state representation is widely used in modern task-oriented dialog systems to model user intentions and slot values. However, a fixed design of domain ontology makes it difficult to extend to new services and APIs. Recent work proposed to
use natural language descriptions to define the domain ontology instead of tag names for each intent or slot, thus offering a dynamic set of schema. In this paper, we conduct in-depth comparative studies to understand the use of natural language description for schema in dialog state tracking. Our discussion mainly covers three aspects: encoder architectures, impact of supplementary training, and effective schema description styles. We introduce a set of newly designed bench-marking descriptions and reveal the model robustness on both homogeneous and heterogeneous description styles in training and evaluation.
Social media companies as well as censorship authorities make extensive use of artificial intelligence (AI) tools to monitor postings of hate speech, celebrations of violence or profanity. Since AI software requires massive volumes of data to train c
omputers, automatic-translation of the online content is usually implemented to compensate for the scarcity of text in some languages. However, machine translation (MT) mistakes are a regular occurrence when translating sentiment-oriented user-generated content (UGC), especially when a low-resource language is involved. In such scenarios, the adequacy of the whole process relies on the assumption that the translation can be evaluated correctly. In this paper, we assess the ability of automatic quality metrics to detect critical machine translation errors which can cause serious misunderstanding of the affect message. We compare the performance of three canonical metrics on meaningless translations as compared to meaningful translations with a critical error that distorts the overall sentiment of the source text. We demonstrate the need for the fine-tuning of automatic metrics to make them more robust in detecting sentiment critical errors.
This work describes analysis of nature and causes of MT errors observed by different evaluators under guidance of different quality criteria: adequacy and comprehension and and a not specified generic mixture of adequacy and fluency. We report result
s for three language pairs and two domains and eleven MT systems. Our findings indicate that and despite the fact that some of the identified phenomena depend on domain and/or language and the following set of phenomena can be considered as generally challenging for modern MT systems: rephrasing groups of words and translation of ambiguous source words and translating noun phrases and and mistranslations. Furthermore and we show that the quality criterion also has impact on error perception. Our findings indicate that comprehension and adequacy can be assessed simultaneously by different evaluators and so that comprehension and as an important quality criterion and can be included more often in human evaluations.
This study aims to determine the direct role
of some factors that contribute to improve the prognosis of the
Cervical and Furcation Perforations throughout application of
bioactive materials using non surgical methods.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
