Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userZero-Shot Information Extraction as a Unified Text-to-Triple Translation
استخراج المعلومات Zero-Shot كترجمة Text-to-triple
489
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
نقوم بإلقاء مجموعة من مهام استخراج المعلومات في إطار ترجمة نصية إلى ثلاثية. بدلا من حل كل مهمة تعتمد على مجموعات البيانات والنماذج الخاصة بالمهام، نقوم بإضفاء الطابع الرسمي على المهمة كترجمة بين نص الإدخال الخاص بمهارات المهام وتصل إلى ثلاث مرات. من خلال اتخاذ المدخلات الخاصة بالمهامة، فإننا تمكن الترجمة المرجعية على المهام من خلال الاستفادة من المعرفة الكامنة التي يحتوي عليها نموذج لغة مدرب مسبقا حول المهمة. نوضح كذلك أن مهمة ما قبل التدريب البسيطة المتمثلة في التنبؤ بالمعلومات العلاجية التي تتوافق مع نص الإدخال هو وسيلة فعالة لإنتاج مخرجات خاصة بالمهام. وهذا يتيح نقل الطلقة الصفرية لإطارنا إلى مهام المصب. ندرس أداء الطلقة الصفرية لهذا الإطار في استخراج المعلومات المفتوح (OIE2016، NYT، WEB، PENN)، تصنيف العلاقة (عدد قليل من الألوان والمستقبلية)، والتحقيق الواقعي (Google-Re و T-Rex). التحويلات النموذجية غير تيهية لمعظم المهام وغالبا ما تكون تنافسية مع طريقة تحت إشراف بالكامل دون الحاجة إلى أي تدريب خاص بمهام. على سبيل المثال، نتفوق بشكل كبير على درجة F1 من استخراج المعلومات المفتوح الخاضعة للإشراف دون الحاجة لاستخدام مجموعة التدريب الخاصة بها.
We cast a suite of information extraction tasks into a text-to-triple translation framework. Instead of solving each task relying on task-specific datasets and models, we formalize the task as a translation between task-specific input text and output triples. By taking the task-specific input, we enable a task-agnostic translation by leveraging the latent knowledge that a pre-trained language model has about the task. We further demonstrate that a simple pre-training task of predicting which relational information corresponds to which input text is an effective way to produce task-specific outputs. This enables the zero-shot transfer of our framework to downstream tasks. We study the zero-shot performance of this framework on open information extraction (OIE2016, NYT, WEB, PENN), relation classification (FewRel and TACRED), and factual probe (Google-RE and T-REx). The model transfers non-trivially to most tasks and is often competitive with a fully supervised method without the need for any task-specific training. For instance, we significantly outperform the F1 score of the supervised open information extraction without needing to use its training set.
References used
https://aclanthology.org/
rate research
Read More
The knowledge of the European silk textile production is a typical case for which the information collected is heterogeneous, spread across many museums and sparse since rarely complete. Knowledge Graphs for this cultural heritage domain, when being
developed with appropriate ontologies and vocabularies, enable to integrate and reconcile this diverse information. However, many of these original museum records still have some metadata gaps. In this paper, we present a zero-shot learning approach that leverages the ConceptNet common sense knowledge graph to predict categorical metadata informing about the silk objects production. We compared the performance of our approach with traditional supervised deep learning-based methods that do require training data. We demonstrate promising and competitive performance for similar datasets and circumstances and the ability to predict sometimes more fine-grained information. Our results can be reproduced using the code and datasets published at https://github.com/silknow/ZSL-KG-silk.
Neural Machine Translation (NMT) approaches employing monolingual data are showing steady improvements in resource-rich conditions. However, evaluations using real-world lowresource languages still result in unsatisfactory performance. This work prop
oses a novel zeroshot NMT modeling approach that learns without the now-standard assumption of a pivot language sharing parallel data with the zero-shot source and target languages. Our approach is based on three stages: initialization from any pre-trained NMT model observing at least the target language, augmentation of source sides leveraging target monolingual data, and learning to optimize the initial model to the zero-shot pair, where the latter two constitute a selflearning cycle. Empirical findings involving four diverse (in terms of a language family, script and relatedness) zero-shot pairs show the effectiveness of our approach with up to +5.93 BLEU improvement against a supervised bilingual baseline. Compared to unsupervised NMT, consistent improvements are observed even in a domain-mismatch setting, attesting to the usability of our method.
Existing approaches for machine translation (MT) mostly translate given text in the source language into the target language and without explicitly referring to information indispensable for producing proper translation. This includes not only inform
ation in other textual elements and modalities than texts in the same document and but also extra-document and non-linguistic information and such as norms and skopos. To design better translation production work-flows and we need to distinguish translation issues that could be resolved by the existing text-to-text approaches and those beyond them. To this end and we conducted an analytic assessment of MT outputs and taking an English-to-Japanese news translation task as a case study. First and examples of translation issues and their revisions were collected by a two-stage post-editing (PE) method: performing minimal PE to obtain translation attainable based on the given textual information and further performing full PE to obtain truly acceptable translation referring to any information if necessary. Then and the collected revision examples were manually analyzed. We revealed dominant issues and information indispensable for resolving them and such as fine-grained style specifications and terminology and domain-specific knowledge and and reference documents and delineating a clear distinction between translation and what text-to-text MT can ultimately attain.
Recent neural text-to-SQL models can effectively translate natural language questions to corresponding SQL queries on unseen databases. Working mostly on the Spider dataset, researchers have proposed increasingly sophisticated solutions to the proble
m. Contrary to this trend, in this paper we focus on simplifications. We begin by building DuoRAT, a re-implementation of the state-of-the-art RAT-SQL model that unlike RAT-SQL is using only relation-aware or vanilla transformers as the building blocks. We perform several ablation experiments using DuoRAT as the baseline model. Our experiments confirm the usefulness of some techniques and point out the redundancy of others, including structural SQL features and features that link the question with the schema.
Zero-shot cross-lingual information extraction (IE) describes the construction of an IE model for some target language, given existing annotations exclusively in some other language, typically English. While the advance of pretrained multilingual enc
oders suggests an easy optimism of train on English, run on any language'', we find through a thorough exploration and extension of techniques that a combination of approaches, both new and old, leads to better performance than any one cross-lingual strategy in particular. We explore techniques including data projection and self-training, and how different pretrained encoders impact them. We use English-to-Arabic IE as our initial example, demonstrating strong performance in this setting for event extraction, named entity recognition, part-of-speech tagging, and dependency parsing. We then apply data projection and self-training to three tasks across eight target languages. Because no single set of techniques performs the best across all tasks, we encourage practitioners to explore various configurations of the techniques described in this work when seeking to improve on zero-shot training.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
