Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEvaluating the Morphosyntactic Well-formedness of Generated Texts
تقييم مورفوسنكتاكي شكل جيد للنصوص المتولدة
308
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
نظم توليد النص في كل مكان في تطبيقات معالجة اللغة الطبيعية.ومع ذلك، فإن تقييم هذه النظم يظل تحديا، خاصة في إعدادات متعددة اللغات.في هذه الورقة، نقترح L'Ambre - مقياس قياسي لتقييم صورة نصية مورفوسنكتاسية من النص باستخدام تحليل التبعية والقواعد المورفوسنكتانية للغة.نقدم طريقة لاستخراج القواعد المختلفة التي تحكم morphosyntax مباشرة من Temessency Treebanks.لمعالجة النواتج الصاخبة من أنظمة جيل النص، نقترح منهجية بسيطة لتدريب المحللين القويين.نظهر فعالية قيادةنا في مهمة الترجمة الآلية من خلال دراسة DIACHRONIC للنظم ترجمة إلى لغات غنية بالمظورة.
Text generation systems are ubiquitous in natural language processing applications. However, evaluation of these systems remains a challenge, especially in multilingual settings. In this paper, we propose L'AMBRE -- a metric to evaluate the morphosyntactic well-formedness of text using its dependency parse and morphosyntactic rules of the language. We present a way to automatically extract various rules governing morphosyntax directly from dependency treebanks. To tackle the noisy outputs from text generation systems, we propose a simple methodology to train robust parsers. We show the effectiveness of our metric on the task of machine translation through a diachronic study of systems translating into morphologically-rich languages.
References used
https://aclanthology.org/
rate research
Read More
Sensitivity of deep-neural models to input noise is known to be a challenging problem. In NLP, model performance often deteriorates with naturally occurring noise, such as spelling errors. To mitigate this issue, models may leverage artificially nois
ed data. However, the amount and type of generated noise has so far been determined arbitrarily. We therefore propose to model the errors statistically from grammatical-error-correction corpora. We present a thorough evaluation of several state-of-the-art NLP systems' robustness in multiple languages, with tasks including morpho-syntactic analysis, named entity recognition, neural machine translation, a subset of the GLUE benchmark and reading comprehension. We also compare two approaches to address the performance drop: a) training the NLP models with noised data generated by our framework; and b) reducing the input noise with external system for natural language correction. The code is released at https://github.com/ufal/kazitext.
User-generated texts include various types of stylistic properties, or noises. Such texts are not properly processed by existing morpheme analyzers or language models based on formal texts such as encyclopedias or news articles. In this paper, we pro
pose a simple morphologically tight-fitting tokenizer (K-MT) that can better process proper nouns, coinages, and internet slang among other types of noise in Korean user-generated texts. We tested our tokenizer by performing classification tasks on Korean user-generated movie reviews and hate speech datasets, and the Korean Named Entity Recognition dataset. Through our tests, we found that K-MT is better fit to process internet slangs, proper nouns, and coinages, compared to a morpheme analyzer and a character-level WordPiece tokenizer.
We propose a method for evaluating the quality of generated text by asking evaluators to count facts, and computing precision, recall, f-score, and accuracy from the raw counts. We believe this approach leads to a more objective and easier to reproduce evaluation. We apply this to the task of medical report summarisation, where measuring objective quality and accuracy is of paramount importance.
We consider the problem of learning to simplify medical texts. This is important because most reliable, up-to-date information in biomedicine is dense with jargon and thus practically inaccessible to the lay audience. Furthermore, manual simplificati
on does not scale to the rapidly growing body of biomedical literature, motivating the need for automated approaches. Unfortunately, there are no large-scale resources available for this task. In this work we introduce a new corpus of parallel texts in English comprising technical and lay summaries of all published evidence pertaining to different clinical topics. We then propose a new metric based on likelihood scores from a masked language model pretrained on scientific texts. We show that this automated measure better differentiates between technical and lay summaries than existing heuristics. We introduce and evaluate baseline encoder-decoder Transformer models for simplification and propose a novel augmentation to these in which we explicitly penalize the decoder for producing jargon'' terms; we find that this yields improvements over baselines in terms of readability.
It is necessary when designing removable partial dentures to be fixed,
stable and well supported for improving patient comfort, and decreasing
damages on soft tissues and dental abutments, the alveolar ridge has a
strong influence in planning for
treatment and interpretations
biomechanical when treated with removable partial dentures, and have
four shapes in the sagittal plane which are horizontal shape, concave
shape, distal ascending shape and descending shape.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
