Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userRoR: Read-over-Read for Long Document Machine Reading Comprehension
ROR: قراءة الزائدة للقراءة لآلة المستندات الطويلة
269
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
حققت النماذج المدربة مسبقا للمحولات، مثل بيرت، نتائج رائعة بشأن فهم القراءة في الآلة. ومع ذلك، نظرا لقيود طول الترميز (E.G.، 512 Tokenspece)، عادة ما يتم تقسيم وثيقة طويلة إلى قطع متعددة يتم قراءتها بشكل مستقل. ينتج عن أن حقل القراءة يقتصر على القطع الفردية دون تعاون المعلومات لفهم قراءة آلات المستندات الطويلة. لمعالجة هذه المشكلة، نقترح ROR، وهي طريقة للقراءة للقراءة، والتي تقوم بتوسيع حقل القراءة من قطعة إلى المستند. على وجه التحديد، يتضمن ROR قارئ قطعة وقارئ مستند. السابق يتوقع أولا مجموعة من الإجابات الإقليمية لكل قطعة، والتي يتم ضغطها بعد ذلك في إصدارا كبيرا مكثفا من المستند الأصلي، مما يضمن ترميزه مرة واحدة. يتنبأ الأخير كذلك بالإجابات العالمية من هذه الوثيقة المكثفة. في النهاية، يتم استخدام استراتيجية التصويت إلى إجمالي الإجابات الإقليمية والعالمية للتنبؤ النهائي. تثبت تجارب واسعة على معيارين Quac و Triviaqa فعالية ROR للحصول على قراءة المستندات الطويلة. والجدير بالذكر أن ROR يحتل المرتبة الأولى على المتصدرين Quac (https://quac.ai/) في وقت التقديم (17 مايو، 2021).
Transformer-based pre-trained models, such as BERT, have achieved remarkable results on machine reading comprehension. However, due to the constraint of encoding length (e.g., 512 WordPiece tokens), a long document is usually split into multiple chunks that are independently read. It results in the reading field being limited to individual chunks without information collaboration for long document machine reading comprehension. To address this problem, we propose RoR, a read-over-read method, which expands the reading field from chunk to document. Specifically, RoR includes a chunk reader and a document reader. The former first predicts a set of regional answers for each chunk, which are then compacted into a highly-condensed version of the original document, guaranteeing to be encoded once. The latter further predicts the global answers from this condensed document. Eventually, a voting strategy is utilized to aggregate and rerank the regional and global answers for final prediction. Extensive experiments on two benchmarks QuAC and TriviaQA demonstrate the effectiveness of RoR for long document reading. Notably, RoR ranks 1st place on the QuAC leaderboard (https://quac.ai/) at the time of submission (May 17th, 2021).
References used
https://aclanthology.org/
rate research
Read More
Adversarial training (AT) as a regularization method has proved its effectiveness on various tasks. Though there are successful applications of AT on some NLP tasks, the distinguishing characteristics of NLP tasks have not been exploited. In this pap
er, we aim to apply AT on machine reading comprehension (MRC) tasks. Furthermore, we adapt AT for MRC tasks by proposing a novel adversarial training method called PQAT that perturbs the embedding matrix instead of word vectors. To differentiate the roles of passages and questions, PQAT uses additional virtual P/Q-embedding matrices to gather the global perturbations of words from passages and questions separately. We test the method on a wide range of MRC tasks, including span-based extractive RC and multiple-choice RC. The results show that adversarial training is effective universally, and PQAT further improves the performance.
Machine reading comprehension is a challenging task especially for querying documents with deep and interconnected contexts. Transformer-based methods have shown advanced performances on this task; however, most of them still treat documents as a fla
t sequence of tokens. This work proposes a new Transformer-based method that reads a document as tree slices. It contains two modules for identifying more relevant text passage and the best answer span respectively, which are not only jointly trained but also jointly consulted at inference time. Our evaluation results show that our proposed method outperforms several competitive baseline approaches on two datasets from varied domains.
This research is based on deconstructing the mechanism of the hidden system of term, in its analytical applications on the two modes (virility / femininity) in Al-Ghathami's cultural criticism. In order to reveal the problems of this monetary term th
at those concerned with cultural criticism in general, or criticism of Al-Ghathami's production in particular. They are problems that can be classified into three types. The first is the paradox / contradiction, which is when there are two texts of Al- Ghathami in one subject, and they are opposite or contradictory, i.e. transcribing one another. The second problem is the problem of play, and it is the text that carries another reading other than the reading of food, which the recipient can memorize through the tools of cultural criticism and its mechanisms themselves. The third type is the problem of sin, which is the text that carries a critical connotation that contradicts and abolishes the cultural criticism mechanism.
Implicit event argument extraction (EAE) is a crucial document-level information extraction task that aims to identify event arguments beyond the sentence level. Despite many efforts for this task, the lack of enough training data has long impeded th
e study. In this paper, we take a new perspective to address the data sparsity issue faced by implicit EAE, by bridging the task with machine reading comprehension (MRC). Particularly, we devise two data augmentation regimes via MRC, including: 1) implicit knowledge transfer, which enables knowledge transfer from other tasks, by building a unified training framework in the MRC formulation, and 2) explicit data augmentation, which can explicitly generate new training examples, by treating MRC models as an annotator. The extensive experiments have justified the effectiveness of our approach --- it not only obtains state-of-the-art performance on two benchmarks, but also demonstrates superior results in a data-low scenario.
In this paper, we present a new verification style reading comprehension dataset named VGaokao from Chinese Language tests of Gaokao. Different from existing efforts, the new dataset is originally designed for native speakers' evaluation, thus requir
ing more advanced language understanding skills. To address the challenges in VGaokao, we propose a novel Extract-Integrate-Compete approach, which iteratively selects complementary evidence with a novel query updating mechanism and adaptively distills supportive evidence, followed by a pairwise competition to push models to learn the subtle difference among similar text pieces. Experiments show that our methods outperform various baselines on VGaokao with retrieved complementary evidence, while having the merits of efficiency and explainability. Our dataset and code are released for further research.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
