يتم تقييم تبسيط النص على مستوى الجملة حاليا باستخدام المقاييس الآلية والتقييم البشري.للتقييم التلقائي، عادة ما يتم توظيف مزيج من المقاييس لتقييم الجوانب المختلفة من التبسيط.مستوى الصف Flesch-Kincaid (FKGL) هو مقياس واحد تم استخدامه بانتظام لقياس قابلية قراءة إخراج النظام.في هذه الورقة، نقول أن FKGL لا ينبغي استخدامها لتقييم أنظمة تبسيط النص.نحن نقدم التحليلات التجريبية على إخراج النظام الأخير الذي يظهر أن درجة FKGL يمكن التلاعب بها بسهولة لتحسين النتيجة بشكل كبير مع تأثير بسيط فقط على مقاييس آلية أخرى (بلو والساري).بدلا من استخدام FKGL، نقترح أن يتم استخدام إحصائيات المكونات، إلى جانب الآخرين، لتحليل posthoc لفهم سلوك النظام.
Sentence-level text simplification is currently evaluated using both automated metrics and human evaluation. For automatic evaluation, a combination of metrics is usually employed to evaluate different aspects of the simplification. Flesch-Kincaid Grade Level (FKGL) is one metric that has been regularly used to measure the readability of system output. In this paper, we argue that FKGL should not be used to evaluate text simplification systems. We provide experimental analyses on recent system output showing that the FKGL score can easily be manipulated to improve the score dramatically with only minor impact on other automated metrics (BLEU and SARI). Instead of using FKGL, we suggest that the component statistics, along with others, be used for posthoc analysis to understand system behavior.
References used
https://aclanthology.org/
The quality of fully automated text simplification systems is not good enough for use in real-world settings; instead, human simplifications are used. In this paper, we examine how to improve the cost and quality of human simplifications by leveragin
Text simplification is a valuable technique. However, current research is limited to sentence simplification. In this paper, we define and investigate a new task of document-level text simplification, which aims to simplify a document consisting of m
Advancements within the field of text simplification (TS) have primarily been within syntactic or lexical simplification. However, conceptual simplification has previously been identified as another field of TS that has the potential to significantly
Large-Scale Multi-Label Text Classification (LMTC) includes tasks with hierarchical label spaces, such as automatic assignment of ICD-9 codes to discharge summaries. Performance of models in prior art is evaluated with standard precision, recall, and
Various temporal knowledge graph (KG) completion models have been proposed in the recent literature. The models usually contain two parts, a temporal embedding layer and a score function derived from existing static KG modeling approaches. Since the