Bidirectional Hierarchical Attention Networks based on Document-level Context for Emotion Cause Extraction

Emotion cause extraction (ECE) aims to extract the causes behind the certain emotion in text. Some works related to the ECE task have been published and attracted lots of attention in recent years. However, these methods neglect two major issues: 1) pay few attentions to the effect of document-level context information on ECE, and 2) lack of sufficient exploration for how to effectively use the annotated emotion clause. For the first issue, we propose a bidirectional hierarchical attention network (BHA) corresponding to the specified candidate cause clause to capture the document-level context in a structured and dynamic manner. For the second issue, we design an emotional filtering module (EF) for each layer of the graph attention network, which calculates a gate score based on the emotion clause to filter the irrelevant information. Combining the BHA and EF, the EF-BHA can dynamically aggregate the contextual information from two directions and filters irrelevant information. The experimental results demonstrate that EF-BHA achieves the competitive performances on two public datasets in different languages (Chinese and English). Moreover, we quantify the effect of context on emotion cause extraction and provide the visualization of the interactions between candidate cause clauses and contexts.

Domain-Specific Japanese ELECTRA Model Using a Small Corpus

Recently, domain shift, which affects accuracy due to differences in data between source and target domains, has become a serious issue when using machine learning methods to solve natural language processing tasks. With additional pretraining and fine-tuning using a target domain corpus, pretraining models such as BERT (Bidirectional Encoder Representations from Transformers) can address this issue. However, the additional pretraining of the BERT model is difficult because it requires significant computing resources. The efficiently learning an encoder that classifies token replacements accurately (ELECTRA) pretraining model replaces the BERT pretraining method's masked language modeling with a method called replaced token detection, which improves the computational efficiency and allows the additional pretraining of the model to a practical extent. Herein, we propose a method for addressing the computational efficiency of pretraining models in domain shift by constructing an ELECTRA pretraining model on a Japanese dataset and additional pretraining this model in a downstream task using a corpus from the target domain. We constructed a pretraining model for ELECTRA in Japanese and conducted experiments on a document classification task using data from Japanese news articles. Results show that even a model smaller than the pretrained model performs equally well.

InFillmore: Frame-Guided Language Generation with Bidirectional Context

We propose a structured extension to bidirectional-context conditional language generation, or infilling,'' inspired by Frame Semantic theory. Guidance is provided through one of two approaches: (1) model fine-tuning, conditioning directly on observed symbolic frames, and (2) a novel extension to disjunctive lexically constrained decoding that leverages frame semantic lexical units. Automatic and human evaluations confirm that frame-guided generation allows for explicit manipulation of intended infill semantics, with minimal loss in distinguishability from human-generated text. Our methods flexibly apply to a variety of use scenarios, and we provide an interactive web demo.

Bidirectional Domain Adaptation Using Weighted Multi-Task Learning

Domain adaption in syntactic parsing is still a significant challenge. We address the issue of data imbalance between the in-domain and out-of-domain treebank typically used for the problem. We define domain adaptation as a Multi-task learning (MTL) problem, which allows us to train two parsers, one for each do-main. Our results show that the MTL approach is beneficial for the smaller treebank. For the larger treebank, we need to use loss weighting in order to avoid a decrease in performance be-low the single task. In order to determine towhat degree the data imbalance between two domains and the domain differences affect results, we also carry out an experiment with two imbalanced in-domain treebanks and show that loss weighting also improves performance in an in-domain setting. Given loss weighting in MTL, we can improve results for both parsers.

Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation

A conventional approach to improving the performance of end-to-end speech translation (E2E-ST) models is to leverage the source transcription via pre-training and joint training with automatic speech recognition (ASR) and neural machine translation (NMT) tasks. However, since the input modalities are different, it is difficult to leverage source language text successfully. In this work, we focus on sequence-level knowledge distillation (SeqKD) from external text-based NMT models. To leverage the full potential of the source language information, we propose backward SeqKD, SeqKD from a target-to-source backward NMT model. To this end, we train a bilingual E2E-ST model to predict paraphrased transcriptions as an auxiliary task with a single decoder. The paraphrases are generated from the translations in bitext via back-translation. We further propose bidirectional SeqKD in which SeqKD from both forward and backward NMT models is combined. Experimental evaluations on both autoregressive and non-autoregressive models show that SeqKD in each direction consistently improves the translation performance, and the effectiveness is complementary regardless of the model capacity.

Explicit Alignment Objectives for Multilingual Bidirectional Encoders

Pre-trained cross-lingual encoders such as mBERT (Devlin et al., 2019) and XLM-R (Conneau et al., 2020) have proven impressively effective at enabling transfer-learning of NLP systems from high-resource languages to low-resource languages. This success comes despite the fact that there is no explicit objective to align the contextual embeddings of words/sentences with similar meanings across languages together in the same space. In this paper, we present a new method for learning multilingual encoders, AMBER (Aligned Multilingual Bidirectional EncodeR). AMBER is trained on additional parallel data using two explicit alignment objectives that align the multilingual representations at different granularities. We conduct experiments on zero-shot cross-lingual transfer learning for different tasks including sequence tagging, sentence retrieval and sentence classification. Experimental results on the tasks in the XTREME benchmark (Hu et al., 2020) show that AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLM-R-large model which has 3.2x the parameters of AMBER. Our code and models are available at http://github.com/junjiehu/amber.

The trend towards vocational education and its relation to vocational awareness among the ninth grade students in Lattakia Governorate

This research aims at identifying the trend towards vocational education and its relation to vocational awareness among the ninth grade students in Lattakia Governorate, and the impact of sex variables and the continuation of school and place of residence in it. In order to achieve the research objectives, the analytical descriptive method was used by designing the first two scales to measure the trend toward vocational education and the second to identify the degree of professional awareness, The research sample consisted of 287 students from the ninth grade in Lattakia Governorate. The research reached the following results: - The trend toward vocational education among the ninth grade students in Lattakia governorate was negative, with an average of 2.39) and a relative weight of (47.8)%.) -The degree of vocational awareness among the ninth grade students in Lattakia governorate was average, with an average of 2.67) and a relative weight of( 53.4)%.) -There were statistically significant differences between the mean scores of the responses of the research sample on the measure of the trend towards vocational education according to the sex variable, in favor of males. -There were no statistically significant differences between the mean scores of the responses of the research sample on the measure of the trend towards vocational education according to the variables of the school (general, special) and place of residence (city, rural).

The Views of Secondary School Teachers Towards the Use of Strategy Inverted learning in Science Teaching Field Study in Lattakia

The aim of the research is to identify the attitudes of secondary school teachers towards the use of the inverted learning strategy in teaching science, and to study differences in their attitudes according to gender variables, years of teaching experience, scientific qualification and computer literacy. The research was based on a descriptive method. A questionnaire was designed and distributed to a random sample of (200) teachers and schools, (187) of which were fully valid and valid for statistical analysis, with a response rate of (93.5%). The research found a number of results, the most important of which is that the attitudes of secondary school teachers towards the use of the inverted learning strategy in teaching science is positive and relatively important (79.9%). Where they have a desire to use this strategy because of their positive implications for the educational process from their point of view, where teachers emphasize that inverted learning contributes to increasing learning time by transforming the process of home learning and solving homework in the classroom, and provides a stimulating learning environment Share learners in the responsibility of learning. The results showed no statistically significant differences between the average scores of teachers in secondary education in Lattakia in their attitudes towards the use of the inverted learning strategy according to the gender variable, while there were statistically significant differences according to the variables of the academic qualification and the years of teaching experience and computer knowledge.

Design Adaptive Controller for Unmanned Aerial Vehicle

The nonlinear model of Unmanned Aerial Vehicle( UAV) has been recognized. Airosim Matlab toolbox has been used to guarantee a simulation model for the Aerosonde.In the first stage, a linearization technique is used to calculate the mathematical model of the UAV at a specific operation point, then PID controller is used to stabilize this linear model. At the final stage, an augmented feedback neural network adaptive controller is applied to stabilize the overall nonlinear system.

The effectiveness of using a computerized interactive program in improving the five senses concepts for the students of Fourth Primary Grade

The current research aims to know the effectiveness of using the suggested computerized interactive programs of the students achievements through designing a learning computerized Unit in science and know its effectiveness in helping the students of fourth primary grade to gain the basic concepts of the five senses, and to achieve this goal a computerized learning program was designed, educational achievement test and an attitudes questionnaire was designed, and the computerized program was applied on two schools (Ibrahim Hanano & Ammar Hassan) from the first cycle schools in Damascus Al-Muhajreen area, and the number of sample members was (74) pupils. The results showed that the suggested learning computerized Unit is effective in helping the students to improve the basic concepts of the five senses, and there was a difference statistically significant between the mean scores of students in the control group (male/ female) and average grades of students of the experimental group (male/ female) for the benefit of the students of the experimental group, in addition to a difference statistically significant between the average grades of students of the control group and the average grades of students of the experimental group due to the variable of method of education for the benefit of the experimental group, and the results showed that 88.24% of students of the fourth primary grade attitudes were positive toward the learning computerized Unit, with no difference between the students attitudes depending on the sex variable, also the results showed that there is no correlating relationship with significant statistical between the fourth grade pupils marks in the achievement test and their marks in the attitudes questionnaire. And one of the most important research proposals was the necessity to cares about the using of computerized interactive educational programs and the employment of it in the educational process and that's for its impact on the achievement and direction.