Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTGIF: Tree-Graph Integrated-Format Parser for Enhanced UD with Two-Stage Generic- to Individual-Language Finetuning
TGIF: Graph Graph مدمج التنسيق المتكامل للمحافظة على UD المعزز مع Finetuning Generic-Langual - اللغة الفردية
360
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
نقدم مساهمتنا في المهمة المشتركة IWPT 2021 بشأن التحليل في التبعيات العالمية المعززة. مكون النظام الرئيسي الخاص بنا هو محلل خرطوم من أشجار الشجرة الهجين التي تدمج (أ) تنبؤات تمتد الأشجار الرسوم البيانية المحسنة مع (ب) حواف بياني إضافية غير موجودة في الأشجار الممتدة. نحن أيضا اعتماد استراتيجية Finetuning حيث نركن أولا محيطا عاما لغة على تسلسل البيانات من جميع اللغات المتاحة، ثم، في خطوة ثانية، Finetune على كل لغة فردية بشكل منفصل. بالإضافة إلى ذلك، نطور مجموعة كاملة من الوحدات الكاملة الخاصة بنا ذات الصلة بالمهمة المشتركة، بما في ذلك التكفيق، تجزئة الجملة، والتوسع الرمز المميز متعدد الكلمات، استنادا إلى نماذج XLM-R المدربة مسبقا ولغة ما قبل التدريب على مستوى الشخصية عارضات ازياء. يوصل تقديمنا إلى ELAS في المتوسط الكلي 89.24 في مجموعة الاختبار. تحتل المرتبة الأولى بين جميع الفرق، مع وجود هامش أكثر من 2 إيلاس المطلق عبر التقديم الأفضل الأداء الأفضل، وأفضل درجة في 16 من أصل 17 لغة.
We present our contribution to the IWPT 2021 shared task on parsing into enhanced Universal Dependencies. Our main system component is a hybrid tree-graph parser that integrates (a) predictions of spanning trees for the enhanced graphs with (b) additional graph edges not present in the spanning trees. We also adopt a finetuning strategy where we first train a language-generic parser on the concatenation of data from all available languages, and then, in a second step, finetune on each individual language separately. Additionally, we develop our own complete set of pre-processing modules relevant to the shared task, including tokenization, sentence segmentation, and multiword token expansion, based on pre-trained XLM-R models and our own pre-training of character-level language models. Our submission reaches a macro-average ELAS of 89.24 on the test set. It ranks top among all teams, with a margin of more than 2 absolute ELAS over the next best-performing submission, and best score on 16 out of 17 languages.
References used
https://aclanthology.org/
rate research
Read More
Coupled with biaffine decoders, transformers have been effectively adapted to text-to-graph transduction and achieved state-of-the-art performance on AMR parsing. Many prior works, however, rely on the biaffine decoder for either or both arc and labe
l predictions although most features used by the decoder may be learned by the transformer already. This paper presents a novel approach to AMR parsing by combining heterogeneous data (tokens, concepts, labels) as one input to a transformer to learn attention, and use only attention matrices from the transformer to predict all elements in AMR graphs (concepts, arcs, labels). Although our models use significantly fewer parameters than the previous state-of-the-art graph parser, they show similar or better accuracy on AMR 2.0 and 3.0.
Knowledge graphs suffer from sparsity which degrades the quality of representations generated by various methods. While there is an abundance of textual information throughout the web and many existing knowledge bases, aligning information across the
se diverse data sources remains a challenge in the literature. Previous work has partially addressed this issue by enriching knowledge graph entities based on hard'' co-occurrence of words present in the entities of the knowledge graphs and external text, while we achieve soft'' augmentation by proposing a knowledge graph enrichment and embedding framework named Edge. Given an original knowledge graph, we first generate a rich but noisy augmented graph using external texts in semantic and structural level. To distill the relevant knowledge and suppress the introduced noise, we design a graph alignment term in a shared embedding space between the original graph and augmented graph. To enhance the embedding learning on the augmented graph, we further regularize the locality relationship of target entity based on negative sampling. Experimental results on four benchmark datasets demonstrate the robustness and effectiveness of Edge in link prediction and node classification.
Abstract For natural language processing systems, two kinds of evidence support the use of text representations from neural language models pretrained'' on large unannotated corpora: performance on application-inspired benchmarks (Peters et al., 2018
, inter alia), and the emergence of syntactic abstractions in those representations (Tenney et al., 2019, inter alia). On the other hand, the lack of grounded supervision calls into question how well these representations can ever capture meaning (Bender and Koller, 2020). We apply novel probes to recent language models--- specifically focusing on predicate-argument structure as operationalized by semantic dependencies (Ivanova et al., 2012)---and find that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning, yielding benefits to natural language understanding (NLU) tasks in the GLUE benchmark. This approach demonstrates the potential for general-purpose (rather than task-specific) linguistic supervision, above and beyond conventional pretraining and finetuning. Several diagnostics help to localize the benefits of our approach.1
We describe the NUIG solution for IWPT 2021 Shared Task of Enhanced Dependency (ED) parsing in multiple languages. For this shared task, we propose and evaluate an End-to-end Seq2seq mBERT-based ED parser which predicts the ED-parse tree of a given i
nput sentence as a relative head-position tag-sequence. Our proposed model is a multitasking neural-network which performs five key tasks simultaneously namely UPOS tagging, UFeat tagging, Lemmatization, Dependency-parsing and ED-parsing. Furthermore we utilise the linguistic typology available in the WALS database to improve the ability of our proposed end-to-end parser to transfer across languages. Results show that our proposed Seq2seq ED-parser performs on par with state-of-the-art ED-parser despite having a much simpler de- sign.
Recent QA with logical reasoning questions requires passage-level relations among the sentences. However, current approaches still focus on sentence-level relations interacting among tokens. In this work, we explore aggregating passage-level clues fo
r solving logical reasoning QA by using discourse-based information. We propose a discourse-aware graph network (DAGN) that reasons relying on the discourse structure of the texts. The model encodes discourse information as a graph with elementary discourse units (EDUs) and discourse relations, and learns the discourse-aware features via a graph network for downstream QA tasks. Experiments are conducted on two logical reasoning QA datasets, ReClor and LogiQA, and our proposed DAGN achieves competitive results. The source code is available at https://github.com/Eleanor-H/DAGN.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
