Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEvaluating Universal Dependency Parser Recovery of Predicate Argument Structure via CompChain Analysis
تقييم استعادة محلل التبعية العالمي بنية الوساطة المسند عبر تحليل كمبيوتر
160
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
إن الانتعاش الدقيق لهيكل الوسائد الواسع من تحليل الاعتماد العالمي (UD) هو أساسي لمهام المصب مثل استخراج الأدوار الدلالية أو تمثيلات الأحداث. تقدم هذه الدراسة على المستحسن، تصنيف التسلسل الهرمي لعلاقات التبعية المستدلة الموجودة داخل تحليل UD. بمثابة دقة تصنيف Compchain بمثابة وكيل لقياس الانتعاش الدقيق لهيكل الوسائد المسند من الجمل مع التضمين. لقد قمنا بتحليل توزيع الملاحظة في Three Treebanks English English، EWT، اللثة والخطوط، وكشف أن هذه Treebanks متناثرة فيما يتعلق بالجمل مع هيكل الوسائد المسند يتضمن تضمين حجة مسند. قمنا بتقييم نماذج خط الأساس SPIPE (V1.2) Conll 2018 (V1.2) الأساس (COMPCHAIN) كمعقل كمبيوتر يعمل بنظام EWT و Gums and Lines UD Treebanks. تشير نتائجنا إلى أن هذه النماذج الأساسية الثلاثة تظهر الأداء الأكثر فقرا في الجمل مع هيكل الوسائد الواسع مع أكثر من مستوى من التضمين؛ استخدمنا Comprains لتوصيف الأخطاء التي تم إجراؤها بواسطة هذه المحللين وتقديم الأمثلة الحالية للضرائب الخاطئة التي تنتجها المحلل المحلل المحدد باستخدام المركبات. لقد قمنا أيضا بتحليل توزيع Comprains في 58 UDBanks UD UDBanks غير الإنجليزية ثم استخدمت Comprains لتقييم نموذج خط الأساس المشترك CONLL'18 لكل من هذه Treebanks. يوضح تحليلنا أن الأداء فيما يتعلق بتصنيف كمبيوتر يحترم ضعيفا ضعيفا فقط مع مقاييس التقييم الرسمية (LAS، MLAS و Blex). نحدد الثغرات في توزيع العقائد في العديد من UD Treebanks، وبالتالي توفير خارطة طريق لكيفية استكمال هذه Treebanks. نستنتج من خلال مناقشة كيفية توفر Comprains منظورا جديدا حول Sparsity بيانات التدريب لمحلل UD، وكذلك دقة تبييض UD الناتج.
Accurate recovery of predicate-argument structure from a Universal Dependency (UD) parse is central to downstream tasks such as extraction of semantic roles or event representations. This study introduces compchains, a categorization of the hierarchy of predicate dependency relations present within a UD parse. Accuracy of compchain classification serves as a proxy for measuring accurate recovery of predicate-argument structure from sentences with embedding. We analyzed the distribution of compchains in three UD English treebanks, EWT, GUM and LinES, revealing that these treebanks are sparse with respect to sentences with predicate-argument structure that includes predicate-argument embedding. We evaluated the CoNLL 2018 Shared Task UDPipe (v1.2) baseline (dependency parsing) models as compchain classifiers for the EWT, GUMS and LinES UD treebanks. Our results indicate that these three baseline models exhibit poorer performance on sentences with predicate-argument structure with more than one level of embedding; we used compchains to characterize the errors made by these parsers and present examples of erroneous parses produced by the parser that were identified using compchains. We also analyzed the distribution of compchains in 58 non-English UD treebanks and then used compchains to evaluate the CoNLL'18 Shared Task baseline model for each of these treebanks. Our analysis shows that performance with respect to compchain classification is only weakly correlated with the official evaluation metrics (LAS, MLAS and BLEX). We identify gaps in the distribution of compchains in several of the UD treebanks, thus providing a roadmap for how these treebanks may be supplemented. We conclude by discussing how compchains provide a new perspective on the sparsity of training data for UD parsers, as well as the accuracy of the resulting UD parses.
References used
https://aclanthology.org/
rate research
Read More
We describe the DCU-EPFL submission to the IWPT 2021 Parsing Shared Task: From Raw Text to Enhanced Universal Dependencies. The task involves parsing Enhanced UD graphs, which are an extension of the basic dependency trees designed to be more facilit
ative towards representing semantic structure. Evaluation is carried out on 29 treebanks in 17 languages and participants are required to parse the data from each language starting from raw strings. Our approach uses the Stanza pipeline to preprocess the text files, XLM-RoBERTa to obtain contextualized token representations, and an edge-scoring and labeling model to predict the enhanced graph. Finally, we run a postprocessing script to ensure all of our outputs are valid Enhanced UD graphs. Our system places 6th out of 9 participants with a coarse Enhanced Labeled Attachment Score (ELAS) of 83.57. We carry out additional post-deadline experiments which include using Trankit for pre-processing, XLM-RoBERTa LARGE, treebank concatenation, and multitask learning between a basic and an enhanced dependency parser. All of these modifications improve our initial score and our final system has a coarse ELAS of 88.04.
The current natural language processing is strongly focused on raising accuracy. The progress comes at a cost of super-heavy models with hundreds of millions or even billions of parameters. However, simple syntactic tasks such as part-of-speech (POS)
tagging, dependency parsing or named entity recognition (NER) do not require the largest models to achieve acceptable results. In line with this assumption we try to minimize the size of the model that jointly performs all three tasks. We introduce ComboNER: a lightweight tool, orders of magnitude smaller than state-of-the-art transformers. It is based on pre-trained subword embeddings and recurrent neural network architecture. ComboNER operates on Polish language data. The model has outputs for POS tagging, dependency parsing and NER. Our paper contains some insights from fine-tuning of the model and reports its overall results.
We describe the NUIG solution for IWPT 2021 Shared Task of Enhanced Dependency (ED) parsing in multiple languages. For this shared task, we propose and evaluate an End-to-end Seq2seq mBERT-based ED parser which predicts the ED-parse tree of a given i
nput sentence as a relative head-position tag-sequence. Our proposed model is a multitasking neural-network which performs five key tasks simultaneously namely UPOS tagging, UFeat tagging, Lemmatization, Dependency-parsing and ED-parsing. Furthermore we utilise the linguistic typology available in the WALS database to improve the ability of our proposed end-to-end parser to transfer across languages. Results show that our proposed Seq2seq ED-parser performs on par with state-of-the-art ED-parser despite having a much simpler de- sign.
Despite recent advances in semantic role labeling propelled by pre-trained text encoders like BERT, performance lags behind when applied to predicates observed infrequently during training or to sentences in new domains. In this work, we investigate
how role labeling performance on low-frequency predicates and out-of-domain data can be further improved by using VerbNet, a verb lexicon that groups verbs into hierarchical classes based on shared syntactic and semantic behavior and defines semantic representations describing relations between arguments. We find that VerbNet classes provide an effective level of abstraction, improving generalization on low-frequency predicates by allowing them to learn from the training examples of other predicates belonging to the same class. We also find that joint training of VerbNet role labeling and predicate disambiguation of VerbNet classes for polysemous verbs leads to improvements in both tasks, naturally supporting the extraction of VerbNet's semantic representations.
We evaluate three leading dependency parser systems from different paradigms on a small yet diverse subset of languages in terms of their accuracy-efficiency Pareto front. As we are interested in efficiency, we evaluate core parsers without pretraine
d language models (as these are typically huge networks and would constitute most of the compute time) or other augmentations that can be transversally applied to any of them. Biaffine parsing emerges as a well-balanced default choice, with sequence-labelling parsing being preferable if inference speed (but not training energy cost) is the priority.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
