Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEverything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction
كل شيء هو كل ما يتطلبه الأمر: استراتيجية متعددة الأحبائية لاستخراج المعلومات الصفرية عبر اللغات
315
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
يصف استخراج المعلومات عبر اللغات الصفرية (IE) بناء نموذج IE لبعض اللغة المستهدفة، بالنظر إلى التعليقات التوضيحية القائمة حصريا في لغة أخرى، عادة باللغة الإنجليزية. في حين أن تقدم اللوائح المتعددة اللغات المحددة مسبقا يشير إلى تفاؤل سهلة للقطار على اللغة الإنجليزية، وتشغيل أي لغة ""، نجد من خلال استكشاف شامل وتمديد التقنيات التي تقودها مجموعة من الأساليب، الجديدة القديمة، إلى أداء أفضل من أي استراتيجية واحدة عبر اللغات على وجه الخصوص. نستكشف التقنيات بما في ذلك إسقاط البيانات والتدريب الذاتي، وكيف تأثير المشفرات المختلفة مسبقا تأثيرها. نستخدم English-to-businal IE مثلي الأولي، مما يدل على أداء قوي في هذا الإعداد لاستخراج الأحداث، والتعرف على الكيان المسمى، ووضع علامات جزء من الكلام، وتحليل التبعية. ثم قم بتطبيق إسقاط البيانات والتدريب الذاتي على ثلاثة مهام عبر ثمانية لغات مستهدفة. نظرا لعدم وجود مجموعة واحدة من التقنيات الأفضل عبر جميع المهام، فإننا نشجع الممارسين على استكشاف تكوينات مختلفة للتقنيات الموضحة في هذا العمل عند السعي لتحسين التدريب على الصفر.
Zero-shot cross-lingual information extraction (IE) describes the construction of an IE model for some target language, given existing annotations exclusively in some other language, typically English. While the advance of pretrained multilingual encoders suggests an easy optimism of train on English, run on any language'', we find through a thorough exploration and extension of techniques that a combination of approaches, both new and old, leads to better performance than any one cross-lingual strategy in particular. We explore techniques including data projection and self-training, and how different pretrained encoders impact them. We use English-to-Arabic IE as our initial example, demonstrating strong performance in this setting for event extraction, named entity recognition, part-of-speech tagging, and dependency parsing. We then apply data projection and self-training to three tasks across eight target languages. Because no single set of techniques performs the best across all tasks, we encourage practitioners to explore various configurations of the techniques described in this work when seeking to improve on zero-shot training.
References used
https://aclanthology.org/
rate research
Read More
We study the power of cross-attention in the Transformer architecture within the context of transfer learning for machine translation, and extend the findings of studies into cross-attention when training from scratch. We conduct a series of experime
nts through fine-tuning a translation model on data where either the source or target language has changed. These experiments reveal that fine-tuning only the cross-attention parameters is nearly as effective as fine-tuning all parameters (i.e., the entire translation model). We provide insights into why this is the case and observe that limiting fine-tuning in this manner yields cross-lingually aligned embeddings. The implications of this finding for researchers and practitioners include a mitigation of catastrophic forgetting, the potential for zero-shot translation, and the ability to extend machine translation models to several new language pairs with reduced parameter storage overhead.
Asking questions about a situation is an inherent step towards understanding it. To this end, we introduce the task of role question generation, which, given a predicate mention and a passage, requires producing a set of questions asking about all po
ssible semantic roles of the predicate. We develop a two-stage model for this task, which first produces a context-independent question prototype for each role and then revises it to be contextually appropriate for the passage. Unlike most existing approaches to question generation, our approach does not require conditioning on existing answers in the text. Instead, we condition on the type of information to inquire about, regardless of whether the answer appears explicitly in the text, could be inferred from it, or should be sought elsewhere. Our evaluation demonstrates that we generate diverse and well-formed questions for a large, broad-coverage ontology of predicates and roles.
Multilingual pre-trained models have achieved remarkable performance on cross-lingual transfer learning. Some multilingual models such as mBERT, have been pre-trained on unlabeled corpora, therefore the embeddings of different languages in the models
may not be aligned very well. In this paper, we aim to improve the zero-shot cross-lingual transfer performance by proposing a pre-training task named Word-Exchange Aligning Model (WEAM), which uses the statistical alignment information as the prior knowledge to guide cross-lingual word prediction. We evaluate our model on multilingual machine reading comprehension task MLQA and natural language interface task XNLI. The results show that WEAM can significantly improve the zero-shot performance.
This paper studies zero-shot cross-lingual transfer of vision-language models. Specifically, we focus on multilingual text-to-video search and propose a Transformer-based model that learns contextual multilingual multimodal embeddings. Under a zero-s
hot setting, we empirically demonstrate that performance degrades significantly when we query the multilingual text-video model with non-English sentences. To address this problem, we introduce a multilingual multimodal pre-training strategy, and collect a new multilingual instructional video dataset (Multi-HowTo100M) for pre-training. Experiments on VTT show that our method significantly improves video search in non-English languages without additional annotations. Furthermore, when multilingual annotations are available, our method outperforms recent baselines by a large margin in multilingual text-to-video search on VTT and VATEX; as well as in multilingual text-to-image search on Multi30K. Our model and Multi-HowTo100M is available at http://github.com/berniebear/Multi-HT100M.
In this paper, we describe our system used for SemEval 2021 Task 7: HaHackathon: Detecting and Rating Humor and Offense. We used a simple fine-tuning approach using different Pre-trained Language Models (PLMs) to evaluate their performance for humor
and offense detection. For regression tasks, we averaged the scores of different models leading to better performance than the original models. We participated in all SubTasks. Our best performing system was ranked 4 in SubTask 1-b, 8 in SubTask 1-c, 12 in SubTask 2, and performed well in SubTask 1-a. We further show comprehensive results using different pre-trained language models which will help as baselines for future work.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
