Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userA Deep Learning System for Automatic Extraction of Typological Linguistic Information from Descriptive Grammars
نظام تعليمي عميق لاستخراج المعلومات اللغوية التلقائية من قواعد النمسات الوصفية
503
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
النمطية اللغوية هي مجال من اللغويات المعنية بتحليل ومقارنة بين اللغات الطبيعية للعالم بناء على ميزاتها اللغوية المعينة. لهذا الغرض، تاريخيا، اعتمدت المنطقة على استخراج يدوية لقيم الميزات اللغوية من الأوصاف النصية للغات. هذا يجعلها مهمة شاقة وطولها باهظة الثمن وهي ملزمة أيضا بسعة الدماغ البشرية. في هذه الدراسة، نقدم نظام تعليمي عميق لمهمة الاستخراج التلقائي للميزات اللغوية من الأوصاف النصية للغات الطبيعية. أولا، يتم تسجيل أوصاف نصية يدويا مع هياكل خاصة تسمى الإطارات الدلالية. يتم تعلم تلك التعليقات التوضيحية من خلال شبكة عصبية متكررة، والتي تستخدم بعد ذلك لتعليق النص غير المشروح. أخيرا، يتم تحويل التعليقات التوضيحية إلى قيم ميزة اللغوية باستخدام وحدة نمطية قائمة على القاعدة. يتم استخدام Tembeddings Word، المستفادة من نص الأغراض العامة، كمصدر رئيسي للمعرفة من قبل الشبكة العصبية المتكررة. قارنا نظام التعلم العميق المقترح للنظام القائم على نظام التعلم الذي تم الإبلاغ عنه مسبقا لنفس المهمة، ويفوز نظام التعلم العميق من حيث درجات F1 بهامش عادل. من المتوقع أن يكون هذا النظام مساهمة مفيدة للجنة التلقائية لقواعد البيانات النموذجية، والتي يتم تطويرها يدويا.
Linguistic typology is an area of linguistics concerned with analysis of and comparison between natural languages of the world based on their certain linguistic features. For that purpose, historically, the area has relied on manual extraction of linguistic feature values from textural descriptions of languages. This makes it a laborious and time expensive task and is also bound by human brain capacity. In this study, we present a deep learning system for the task of automatic extraction of linguistic features from textual descriptions of natural languages. First, textual descriptions are manually annotated with special structures called semantic frames. Those annotations are learned by a recurrent neural network, which is then used to annotate un-annotated text. Finally, the annotations are converted to linguistic feature values using a separate rule based module. Word embeddings, learned from general purpose text, are used as a major source of knowledge by the recurrent neural network. We compare the proposed deep learning system to a previously reported machine learning based system for the same task, and the deep learning system wins in terms of F1 scores with a fair margin. Such a system is expected to be a useful contribution for the automatic curation of typological databases, which otherwise are manually developed.
References used
https://aclanthology.org/
rate research
Read More
Recent information extraction approaches have relied on training deep neural models. However, such models can easily overfit noisy labels and suffer from performance degradation. While it is very costly to filter noisy labels in large learning resour
ces, recent studies show that such labels take more training steps to be memorized and are more frequently forgotten than clean labels, therefore are identifiable in training. Motivated by such properties, we propose a simple co-regularization framework for entity-centric information extraction, which consists of several neural models with identical structures but different parameter initialization. These models are jointly optimized with the task-specific losses and are regularized to generate similar predictions based on an agreement loss, which prevents overfitting on noisy labels. Extensive experiments on two widely used but noisy benchmarks for information extraction, TACRED and CoNLL03, demonstrate the effectiveness of our framework. We release our code to the community for future research.
We curated WikiPII, an automatically labeled dataset composed of Wikipedia biography pages, annotated for personal information extraction. Although automatic annotation can lead to a high degree of label noise, it is an inexpensive process and can ge
nerate large volumes of annotated documents. We trained a BERT-based NER model with WikiPII and showed that with an adequately large training dataset, the model can significantly decrease the cost of manual information extraction, despite the high level of label noise. In a similar approach, organizations can leverage text mining techniques to create customized annotated datasets from their historical data without sharing the raw data for human annotation. Also, we explore collaborative training of NER models through federated learning when the annotation is noisy. Our results suggest that depending on the level of trust to the ML operator and the volume of the available data, distributed training can be an effective way of training a personal information identifier in a privacy-preserved manner. Research material is available at https://github.com/ratmcu/wikipiifed.
Information extraction and question answering have the potential to introduce a new paradigm for how machine learning is applied to criminal law. Existing approaches generally use tabular data for predictive metrics. An alternative approach is needed
for matters of equitable justice, where individuals are judged on a case-by-case basis, in a process involving verbal or written discussion and interpretation of case factors. Such discussions are individualized, but they nonetheless rely on underlying facts. Information extraction can play an important role in surfacing these facts, which are still important to understand. We analyze unsupervised, weakly supervised, and pre-trained models' ability to extract such factual information from the free-form dialogue of California parole hearings. With a few exceptions, most F1 scores are below 0.85. We use this opportunity to highlight some opportunities for further research for information extraction and question answering. We encourage new developments in NLP to enable analysis and review of legal cases to be done in a post-hoc, not predictive, manner.
Document-level relation extraction aims to identify relations between entities in a whole document. Prior efforts to capture long-range dependencies have relied heavily on implicitly powerful representations learned through (graph) neural networks, w
hich makes the model less transparent. To tackle this challenge, in this paper, we propose LogiRE, a novel probabilistic model for document-level relation extraction by learning logic rules. LogiRE treats logic rules as latent variables and consists of two modules: a rule generator and a relation extractor. The rule generator is to generate logic rules potentially contributing to final predictions, and the relation extractor outputs final predictions based on the generated logic rules. Those two modules can be efficiently optimized with the expectation-maximization (EM) algorithm. By introducing logic rules into neural networks, LogiRE can explicitly capture long-range dependencies as well as enjoy better interpretation. Empirical results show that significantly outperforms several strong baselines in terms of relation performance and logical consistency. Our code is available at https://github.com/rudongyu/LogiRE.
relation extraction systems have made extensive use of features generated
by linguistic analysis modules. Errors in these features lead to errors of
relation detection and classification. In this work, we depart from these
traditional approaches w
ith complicated feature engineering by introducing
a convolutional neural network for relation extraction that automatically
learns features from sentences and minimizes the dependence on external
toolkits and resources. Our model takes advantages of multiple window
sizes for filters and pre-trained word embeddings as an initializer on a nonstatic
architecture to improve the performance.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
