Learning Logic Rules for Document-Level Relation Extraction

Document-level relation extraction aims to identify relations between entities in a whole document. Prior efforts to capture long-range dependencies have relied heavily on implicitly powerful representations learned through (graph) neural networks, which makes the model less transparent. To tackle this challenge, in this paper, we propose LogiRE, a novel probabilistic model for document-level relation extraction by learning logic rules. LogiRE treats logic rules as latent variables and consists of two modules: a rule generator and a relation extractor. The rule generator is to generate logic rules potentially contributing to final predictions, and the relation extractor outputs final predictions based on the generated logic rules. Those two modules can be efficiently optimized with the expectation-maximization (EM) algorithm. By introducing logic rules into neural networks, LogiRE can explicitly capture long-range dependencies as well as enjoy better interpretation. Empirical results show that significantly outperforms several strong baselines in terms of relation performance and logical consistency. Our code is available at https://github.com/rudongyu/LogiRE.

Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits

Training data for machine translation (MT) is often sourced from a multitude of large corpora that are multi-faceted in nature, e.g. containing contents from multiple domains or different levels of quality or complexity. Naturally, these facets do not occur with equal frequency, nor are they equally important for the test scenario at hand. In this work, we propose to optimize this balance jointly with MT model parameters to relieve system developers from manual schedule design. A multi-armed bandit is trained to dynamically choose between facets in a way that is most beneficial for the MT system. We evaluate it on three different multi-facet applications: balancing translationese and natural training data, or data from multiple domains or multiple language pairs. We find that bandit learning leads to competitive MT systems across tasks, and our analysis provides insights into its learned strategies and the underlying data sets.

Self-Training using Rules of Grammar for Few-Shot NLU

We tackle the problem of self-training networks for NLU in low-resource environment---few labeled data and lots of unlabeled data. The effectiveness of self-training is a result of increasing the amount of training data while training. Yet it becomes less effective in low-resource settings due to unreliable labels predicted by the teacher model on unlabeled data. Rules of grammar, which describe the grammatical structure of data, have been used in NLU for better explainability. We propose to use rules of grammar in self-training as a more reliable pseudo-labeling mechanism, especially when there are few labeled data. We design an effective algorithm that constructs and expands rules of grammar without human involvement. Then we integrate the constructed rules as a pseudo-labeling mechanism into self-training. There are two possible scenarios regarding data distribution: it is unknown or known in prior to training. We empirically demonstrate that our approach substantially outperforms the state-of-the-art methods in three benchmark datasets for both scenarios.

When is Wall a Pared and when a Muro?: Extracting Rules Governing Lexical Selection

Learning fine-grained distinctions between vocabulary items is a key challenge in learning a new language. For example, the noun wall'' has different lexical manifestations in Spanish -- pared'' refers to an indoor wall while muro'' refers to an outside wall. However, this variety of lexical distinction may not be obvious to non-native learners unless the distinction is explained in such a way. In this work, we present a method for automatically identifying fine-grained lexical distinctions, and extracting rules explaining these distinctions in a human- and machine-readable format. We confirm the quality of these extracted rules in a language learning setup for two languages, Spanish and Greek, where we use the rules to teach non-native speakers when to translate a given ambiguous word into its different possible translations.

CLaC-BP at SemEval-2021 Task 8: SciBERT Plus Rules for MeasEval

This paper explains the design of a heterogeneous system that ranked eighth in competition in SemEval2021 Task 8. We analyze ablation experiments and demonstrate how the system components, namely tokenizer, unit identifier, modifier classifier, and language model, affect the overall score. We compare our results to similar experiments from the literature and introduce a grouping algorithm developed in the post-evaluation phase that increased our system's overall score, hypothetically elevating our competition rank from eight to six.

Teach the Rules, Provide the Facts: Targeted Relational-knowledge Enhancement for Textual Inference

We present InferBert, a method to enhance transformer-based inference models with relevant relational knowledge. Our approach facilitates learning generic inference patterns requiring relational knowledge (e.g. inferences related to hypernymy) during training, while injecting on-demand the relevant relational facts (e.g. pangolin is an animal) at test time. We apply InferBERT to the NLI task over a diverse set of inference types (hypernymy, location, color, and country of origin), for which we collected challenge datasets. In this setting, InferBert succeeds to learn general inference patterns, from a relatively small number of training instances, while not hurting performance on the original NLI data and substantially outperforming prior knowledge enhancement models on the challenge data. It further applies its inferences successfully at test time to previously unobserved entities. InferBert is computationally more efficient than most prior methods, in terms of number of parameters, memory consumption and training time.

Anti-Money Laundering using Data Mining techniques

This research presents literature review on using Artificial intelligence and Data Mining techniques in Anti Money Laundering systems. We compare many methodologies used in different research papers with the purpose of shedding some light on real life applications using Artificial intelligence

GRB Grobner rules applications in Diophantus systems

The topic of the study in this research is one of the important topics in Number Theory, where did we get into man techniques and systems related to Diophantus equations.

Context of situation in the old linguistic lesson "significance and origin"

Meaning precision and fulfillment has been the sole aim of any researcher in language, and since meaning is the outcome of grammatical structure in one specific context, that researcher must not prefer one to the other In other words, all of the grammatical aspects fall in the trap of one controversial relation when it comes to meaning interpretation Therefore, I have tackled the concept of ' situation context ' in my research and the related terminologies and its impact in the field of language , the impact of the religious factor in attracting the old scientists to the significance of grammatical from and context, the outcome of both and to what extent those scientists have relied upon them when interpreting Qur'anic text, lines of poetry only to figure out of them some new rules.

Improving Efficiency Apriori Algorithm by Reduction of candidate itemsets

Association Rules is an important field in Data Mining, which is used to discover useful knowledge from a massive databases. Association Rules have been used to extract the information from the database transactions, and Apriori Algorithm is a practical application for Association Rules and it is used to find frequent itemsets from database transactions. In this paper, we present a new improving on Apriori Algorithm by reduction generating of candidate itemsets and this leads to improving efficiency Apriori Algorithm.