Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAMR Parsing with Action-Pointer Transformer
تحليل عمرو مع محول الحركة المؤشر
346
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تعد تحليل التمثيل المعنى التجريدي مهمة التنبؤ بالسجن إلى الرسم حيث لا تتماشى العقد المستهدفة بشكل صريح إلى رموز الجملة. ومع ذلك، نظرا لأن عقد الرسوم البيانية تستند بشكل دلون على أساس واحد أو أكثر من رموز الجملة، يمكن استخلاص المحاذاة الضمنية. تعمل المحللون المستقلون على الانتقال عبر الجملة من اليسار إلى اليمين، والتقاط هذا التحيز الاستقرائي عبر المحاذاة بتكلفة التعبير المحدود. في هذا العمل، نقترح نظام يستند إلى الانتقال الذي يجمع بين الصعب على الجمل مع آلية مؤشر الإجراءات ذات الجانب المستهدف لتحديد الرموز المصدر من تمثيلات العقدة ومحاذاة العناوين. نقوم بالنماذج التحولات وكذلك آلية المؤشر من خلال تعديلات واضحة داخل بنية محول واحدة. يتم تشفير معلومات هيكل الدولة ومحلل الرسم البياني بكفاءة باستخدام رؤوس الانتباه. نظظ أن نهج مؤشر العمل لدينا يؤدي إلى زيادة التعبير عن التعبير والكتابة مكاسب كبيرة (+1.6 نقطة) مقابل أفضل محلل عمرو على الانتقال في ظروف مماثلة للغاية. أثناء استخدام أي رسوم إعادة التصنيف الرسم البياني، فإن طرازنا الفردي ينتج عنه ثاني أفضل نقاط Smatch في AMR 2.0 (81.8)، والذي تم تحسينه إلى 83.4 مع بيانات الفضة والكشف عن الفرم.
Abstract Meaning Representation parsing is a sentence-to-graph prediction task where target nodes are not explicitly aligned to sentence tokens. However, since graph nodes are semantically based on one or more sentence tokens, implicit alignments can be derived. Transition-based parsers operate over the sentence from left to right, capturing this inductive bias via alignments at the cost of limited expressiveness. In this work, we propose a transition-based system that combines hard-attention over sentences with a target-side action pointer mechanism to decouple source tokens from node representations and address alignments. We model the transitions as well as the pointer mechanism through straightforward modifications within a single Transformer architecture. Parser state and graph structure information are efficiently encoded using attention heads. We show that our action-pointer approach leads to increased expressiveness and attains large gains (+1.6 points) against the best transition-based AMR parser in very similar conditions. While using no graph re-categorization, our single model yields the second best Smatch score on AMR 2.0 (81.8), which is further improved to 83.4 with silver data and ensemble decoding.
References used
https://aclanthology.org/
rate research
Read More
We study multilingual AMR parsing from the perspective of knowledge distillation, where the aim is to learn and improve a multilingual AMR parser by using an existing English parser as its teacher. We constrain our exploration in a strict multilingua
l setting: there is but one model to parse all different languages including English. We identify that noisy input and precise output are the key to successful distillation. Together with extensive pre-training, we obtain an AMR parser whose performances surpass all previously published results on four different foreign languages, including German, Spanish, Italian, and Chinese, by large margins (up to 18.8 Smatch points on Chinese and on average 11.3 Smatch points). Our parser also achieves comparable performance on English to the latest state-of-the-art English-only parser.
Lacking sufficient human-annotated data is one main challenge for abstract meaning representation (AMR) parsing. To alleviate this problem, previous works usually make use of silver data or pre-trained language models. In particular, one recent seq-t
o-seq work directly fine-tunes AMR graph sequences on the encoder-decoder pre-trained language model and achieves new state-of-the-art results, outperforming previous works by a large margin. However, it makes the decoding relatively slower. In this work, we investigate alternative approaches to achieve competitive performance at faster speeds. We propose a simplified AMR parser and a pre-training technique for the effective usage of silver data. We conduct extensive experiments on the widely used AMR2.0 dataset and the results demonstrate that our Transformer-based AMR parser achieves the best performance among the seq2graph-based models. Furthermore, with silver data, our model achieves competitive results with the SOTA model, and the speed is an order of magnitude faster. Detailed analyses are conducted to gain more insights into our proposed model and the effectiveness of the pre-training technique.
In a current experiment we were testing CommonGen dataset for structure-to-text task from GEM living benchmark with the constraint based POINTER model. POINTER represents a hybrid architecture, combining insertion-based and transformer paradigms, pre
dicting the token and the insertion position at the same time. The text is therefore generated gradually in a parallel non-autoregressive manner, given the set of keywords. The pretrained model was fine-tuned on a training split of the CommonGen dataset and the generation result was compared to the validation and challenge splits. The received metrics outputs, which measure lexical equivalence, semantic similarity and diversity, are discussed in details in a present system description.
We propose the Recursive Non-autoregressive Graph-to-Graph Transformer architecture (RNGTr) for the iterative refinement of arbitrary graphs through the recursive application of a non-autoregressive Graph-to-Graph Transformer and apply it to syntacti
c dependency parsing. We demonstrate the power and effectiveness of RNGTr on several dependency corpora, using a refinement model pre-trained with BERT. We also introduce Syntactic Transformer (SynTr), a non-recursive parser similar to our refinement model. RNGTr can improve the accuracy of a variety of initial parsers on 13 languages from the Universal Dependencies Treebanks, English and Chinese Penn Treebanks, and the German CoNLL2009 corpus, even improving over the new state-of-the-art results achieved by SynTr, significantly improving the state-of-the-art for all corpora tested.
Predicting linearized Abstract Meaning Representation (AMR) graphs using pre-trained sequence-to-sequence Transformer models has recently led to large improvements on AMR parsing benchmarks. These parsers are simple and avoid explicit modeling of str
ucture but lack desirable properties such as graph well-formedness guarantees or built-in graph-sentence alignments. In this work we explore the integration of general pre-trained sequence-to-sequence language models and a structure-aware transition-based approach. We depart from a pointer-based transition system and propose a simplified transition set, designed to better exploit pre-trained language models for structured fine-tuning. We also explore modeling the parser state within the pre-trained encoder-decoder architecture and different vocabulary strategies for the same purpose. We provide a detailed comparison with recent progress in AMR parsing and show that the proposed parser retains the desirable properties of previous transition-based approaches, while being simpler and reaching the new parsing state of the art for AMR 2.0, without the need for graph re-categorization.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
