SHAPE: Shifted Absolute Position Embedding for Transformers

Position representation is crucial for building position-aware representations in Transformers. Existing position representations suffer from a lack of generalization to test data with unseen lengths or high computational cost. We investigate shifted absolute position embedding (SHAPE) to address both issues. The basic idea of SHAPE is to achieve shift invariance, which is a key property of recent successful position representations, by randomly shifting absolute positions during training. We demonstrate that SHAPE is empirically comparable to its counterpart while being simpler and faster.

Explore Better Relative Position Embeddings from Encoding Perspective for Transformer Models

Relative position embedding (RPE) is a successful method to explicitly and efficaciously encode position information into Transformer models. In this paper, we investigate the potential problems in Shaw-RPE and XL-RPE, which are the most representative and prevalent RPEs, and propose two novel RPEs called Low-level Fine-grained High-level Coarse-grained (LFHC) RPE and Gaussian Cumulative Distribution Function (GCDF) RPE. LFHC-RPE is an improvement of Shaw-RPE, which enhances the perception ability at medium and long relative positions. GCDF-RPE utilizes the excellent properties of the Gaussian function to amend the prior encoding mechanism in XL-RPE. Experimental results on nine authoritative datasets demonstrate the effectiveness of our methods empirically. Furthermore, GCDF-RPE achieves the best overall performance among five different RPEs.

Modeling Users and Online Communities for Abuse Detection: A Position on Ethics and Explainability

Abuse on the Internet is an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse across various platforms. The psychological effects of abuse on individuals can be profound and lasting. Consequently, over the past few years, there has been a substantial research effort towards automated abusive language detection in the field of NLP. In this position paper, we discuss the role that modeling of users and online communities plays in abuse detection. Specifically, we review and analyze the state of the art methods that leverage user or community information to enhance the understanding and detection of abusive language. We then explore the ethical challenges of incorporating user and community information, laying out considerations to guide future research. Finally, we address the topic of explainability in abusive language detection, proposing properties that an explainable method should aim to exhibit. We describe how user and community information can facilitate the realization of these properties and discuss the effective operationalization of explainability in view of the properties.

Transformer with Syntactic Position Encoding for Machine Translation

It has been widely recognized that syntax information can help end-to-end neural machine translation (NMT) systems to achieve better translation. In order to integrate dependency information into Transformer based NMT, existing approaches either exploit words' local head-dependent relations, ignoring their non-local neighbors carrying important context; or approximate two words' syntactic relation by their relative distance on the dependency tree, sacrificing exactness. To address these issues, we propose global positional encoding for dependency tree, a new scheme that facilitates syntactic relation modeling between any two words with keeping exactness and without immediate neighbor constraint. Experiment results on NC11 German→English, English→German and WMT English→German datasets show that our approach is more effective than the above two strategies. In addition, our experiments quantitatively show that compared with higher layers, lower layers of the model are more proper places to incorporate syntax information in terms of each layer's preference to the syntactic pattern and the final performance.

Utterance Position-Aware Dialogue Act Recognition

This study proposes an utterance position-aware approach for a neural network-based dialogue act recognition (DAR) model, which incorporates positional encoding for utterance's absolute or relative position. The proposed approach is inspired by the observation that some dialogue acts have tendencies of occurrence positions. The evaluations on the Switchboard corpus show that the proposed positional encoding of utterances statistically significantly improves the performance of DAR.

Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphs

We present Graformer, a novel Transformer-based encoder-decoder architecture for graph-to-text generation. With our novel graph self-attention, the encoding of a node relies on all nodes in the input graph - not only direct neighbors - facilitating the detection of global patterns. We represent the relation between two nodes as the length of the shortest path between them. Graformer learns to weight these node-node relations differently for different attention heads, thus virtually learning differently connected views of the input graph. We evaluate Graformer on two popular graph-to-text generation benchmarks, AGENDA and WebNLG, where it achieves strong performance while using many fewer parameters than other approaches.

Studying And Analyzing the modulators and demodulators of PCM

Modulating signals of transmission stations to reception stations is a key factor to guarantee the best possible transmission and reception of these signals .Digital modulation represents a huge evolution in communication field and modulation, which used to depend on analog signal modulation of one parameter-Amplitude . frequency or phase. Digital modulation depends on transforming the transmitted data signal (Bits) and then sending it as samples, and changed back into an analog signals in reception station . In digital systems, digital data are transformed into analog data in the transmitter and does the reverse in the receiver. In digital transmission, on the other hand, as in wired local area networks (WLAN), Digital data are transmitted in their digital state.

Relationship between craniofacial morphology and cervical vertebral posture in patients with skeletal Class II malocclusion

The aim of this study was to investigate the relationship between cervical vertebral posture in patients with skeletal Class II malocclusion by using lateral cephalometric radiographs were taken by natural head position.

Integration of astronomical and geodetic methods to obtain coordinates and optimum azimuth

In this research we will examine the method of calculating azimuth and latitude based on the parameters and mathematical equations for the movement of the earth، stars and astronomical equipment.

The Moral Position of the Robert Maltos Population Theory

This research attempts to shed light on the issue of growing or uncontrolled population growth, especially from the point of view of Robert Maltus as one of the inhabitants who left their silence in this area. This study also addresses several key aspects: First, the reasons behind population growth such as migration, low mortality due to improved health care, attention to women's reproductive health and availability of medication. Second: the relationship between both the population increase and the food problem, from the point of view of Maltos, who believes that there is a direct relationship between the two variables, the more the population has worsened the problem of food. Thirdly, reference is made to the main effects that unbalanced population growth may have on the environment on the one hand, such as continued logging, population expansion, the need for fresh drinking water, pollution of air, water, soil, and the inability to absorb waste. On the social side, poverty, unemployment and the low social level, . The most prominent solutions presented by Maltos to solve the population problem include ethical barriers and natural contraindications. Fifth: To review some attitudes on the population issue such as the theory of Thomas Sadler, James Stewart, Herbert Spencer, Karl Marx, and to indicate the extent of intersection or difference with the theory of Maltos.