Unsupervised Relation Extraction: A Variational Autoencoder Approach

Unsupervised relation extraction works by clustering entity pairs that have the same relations in the text. Some existing variational autoencoder (VAE)-based approaches train the relation extraction model as an encoder that generates relation classifications. A decoder is trained along with the encoder to reconstruct the encoder input based on the encoder-generated relation classifications. These classifications are a latent variable so they are required to follow a pre-defined prior distribution which results in unstable training. We propose a VAE-based unsupervised relation extraction technique that overcomes this limitation by using the classifications as an intermediate variable instead of a latent variable. Specifically, classifications are conditioned on sentence input, while the latent variable is conditioned on both the classifications and the sentence input. This allows our model to connect the decoder with the encoder without putting restrictions on the classification distribution; which improves training stability. Our approach is evaluated on the NYT dataset and outperforms state-of-the-art methods.

Robust Retrieval Augmented Generation for Zero-shot Slot Filling

Automatically inducing high quality knowledge graphs from a given collection of documents still remains a challenging problem in AI. One way to make headway for this problem is through advancements in a related task known as slot filling. In this task, given an entity query in form of [Entity, Slot, ?], a system is asked to fill' the slot by generating or extracting the missing value exploiting evidence extracted from relevant passage(s) in the given document collection. The recent works in the field try to solve this task in an end-to-end fashion using retrieval-based language models. In this paper, we present a novel approach to zero-shot slot filling that extends dense passage retrieval with hard negatives and robust training procedures for retrieval augmented generation models. Our model reports large improvements on both T-REx and zsRE slot filling datasets, improving both passage retrieval and slot value generation, and ranking at the top-1 position in the KILT leaderboard. Moreover, we demonstrate the robustness of our system showing its domain adaptation capability on a new variant of the TACRED dataset for slot filling, through a combination of zero/few-shot learning. We release the source code and pre-trained models.

Learning Sparse Sentence Encoding without Supervision: An Exploration of Sparsity in Variational Autoencoders

It has been long known that sparsity is an effective inductive bias for learning efficient representation of data in vectors with fixed dimensionality, and it has been explored in many areas of representation learning. Of particular interest to this work is the investigation of the sparsity within the VAE framework which has been explored a lot in the image domain, but has been lacking even a basic level of exploration in NLP. Additionally, NLP is also lagging behind in terms of learning sparse representations of large units of text e.g., sentences. We use the VAEs that induce sparse latent representations of large units of text to address the aforementioned shortcomings. First, we move in this direction by measuring the success of unsupervised state-of-the-art (SOTA) and other strong VAE-based sparsification baselines for text and propose a hierarchical sparse VAE model to address the stability issue of SOTA. Then, we look at the implications of sparsity on text classification across 3 datasets, and highlight a link between performance of sparse latent representations on downstream tasks and its ability to encode task-related information.

Simple induction of (deterministic) probabilistic finite-state automata for phonotactics by stochastic gradient descent

We introduce a simple and highly general phonotactic learner which induces a probabilistic finite-state automaton from word-form data. We describe the learner and show how to parameterize it to induce unrestricted regular languages, as well as how to restrict it to certain subregular classes such as Strictly k-Local and Strictly k-Piecewise languages. We evaluate the learner on its ability to learn phonotactic constraints in toy examples and in datasets of Quechua and Navajo. We find that an unrestricted learner is the most accurate overall when modeling attested forms not seen in training; however, only the learner restricted to the Strictly Piecewise language class successfully captures certain nonlocal phonotactic constraints. Our learner serves as a baseline for more sophisticated methods.

Extractive Opinion Summarization in Quantized Transformer Spaces

Abstract We present the Quantized Transformer (QT), an unsupervised system for extractive opinion summarization. QT is inspired by Vector- Quantized Variational Autoencoders, which we repurpose for popularity-driven summarization. It uses a clustering interpretation of the quantized space and a novel extraction algorithm to discover popular opinions among hundreds of reviews, a significant step towards opinion summarization of practical scope. In addition, QT enables controllable summarization without further training, by utilizing properties of the quantized space to extract aspect-specific summaries. We also make publicly available Space, a large-scale evaluation benchmark for opinion summarizers, comprising general and aspect-specific summaries for 50 hotels. Experiments demonstrate the promise of our approach, which is validated by human studies where judges showed clear preference for our method over competitive baselines.

Adversarial Attacks on Deep Learning Systems

Deep learning is at the heart of the current rise of artificial intelligence. In the field of Computer Vision, it has become the workhorse for applications ranging from self-driving cars to surveillance and security. Whereas deep neural networks have demonstrated phenomenal success (often beyond human capabilities) in solving complex problems, recent studies show that they are vulnerable to adversarial attacks in the form of subtle perturbations to inputs that lead a model to predict incorrect outputs. For images, such perturbations are often too small to be perceptible, yet they completely fool the deep learning models. Adversarial attacks pose a serious threat to the success of deep learning in practice. This fact has recently lead to a large influx of contributions in this direction. This article presents a survey on adversarial attacks on deep learning in Computer Vision. We review the works that design adversarial attacks, analyze the existence of such attacks and propose defenses against them

Effect of addition steel fiber abstracted from recycled tires upon the concrete compressive strength

Recently worldwide researches have been devoted to the use of steel fibers recovered from used tires in concrete. In Syria the amount of recovered steel from used tires is estimated about 6000 tons/year. For this purpose a bead wire having a diameter of 0.8 mm from burnt tires was extracted and used. Three mixes with cement content 300- 350 -400kg/m3 were produced incorporating three different volumes of fiber 0.5%, 1.0%, and 1.5% and three different length 30-40-60 mm. The concrete obtained by adding these fibers evidenced a satisfactory improvement of the fragile matrix mostly in terms of toughness and post cracking behavior On the other hand it was improvements in compressive strength by steel fiber inclusion the interesting results confirm the promising application of concrete reinforced with steel fibers extracted from used tires in aircraft pavement hydraulic structures and ground slab in fabrics.

MATLAB& C-Code–Based Modelling& Simulation of Adaptive Power Supply System For Fuel Economy Improvement of Petrol Vehicles

The present paper provides a mathematical dynamic modelling and analysis of a newly developed electrical power supply system for petrol cars. The main goal of this research work is to improve the economic operation of the car’s electric power supply aiming at fuel economy improvement and for less gases emission. The proposed scheme has been studied and investigated using MATLAB/SIMULINK(R) toolbox and C language. The dynamic behaviour of the proposed system has been analysed for two different driving cycles and the results showed a fuel economy improvement in the investigated two cases. It is shown that the proposed approach is very superior to the conventional one, which is widely used in most of public cars. However, it also shows very encouraging results in terms of fuel save and less gas emission. The main advantage of the method proposed is its applicability to any first or second hand low cost cars.

نظام الكشف التلقائي عن أرقام لوحات السيارات باللغة العربية

في هذا البحث ، تم اقتراح طريقة جديدة منخفضة التكلفة للتعرف على لوحات ترخيص المركبات (LPR) والتي يمكن استخدامها بسهولة في لوحات أخرى. تم استخدام تقنية تجزئة LP جديدة مع ثلاث مجموعات من نواقل الميزات مع مطابقة القالب لتشكيل الوحدتين الرئيسيتين: وحدة توطين لوحة الترخيص ووحدة LPR. تم اختبار هذه الطريقة على أكثر من 238 صورة مركبة مأخوذة من مشاهد مختلفة بخطوط وخلفيات مختلفة من دولتين عربيتين. كانت دقة التجزئة للنظام المنفذ 97.5٪ مع دقة التعرف على 99٪ للصور المشوهة إلى حد ما. يوضح النموذج المقدم أنه على الرغم من التأثير السلبي للظلال والشقوق والأوساخ وفصل الشخصيات ، أظهر النظام معدل نجاح إجمالي بنسبة 92٪ في توطين الألواح و 95٪ لتجزئة اللوحات و 92٪ للتعرف على البلد والمدينة و 99 ٪ لتجزئة الرقم والتمييز. أدى الجمع بين جميع المعدلات إلى دقة نظام كلية بلغت 93٪. مقارنة بالعديد من أنظمة LPR المتطورة ، يستخدم هذا النظام المطور حديثًا 3 مجموعات تدريب صغيرة تقلل من أوقات تشغيل الحل المقترح إلى أقل من 5 ثوانٍ باستخدام MATLAB R2008A الذي يعمل على Compaq 8510W مع ذاكرة وصول عشوائي (RAM) 4 جيجا. النتائج قابلة للمقارنة ، وفي بعض الحالات تكون أفضل مع ظروف مقيدة مثل مكان الانحراف وحجم اللوحة والإضاءة والخلفية.

Use Of A New Developed Micro-Pulse-Like Reactor For The Kinetic Catalytic Measurements Of NO, NOX Removal From Car Exhaust Gases On The New Catalyst (BZ Ag2O, Al2O3-MoO3- Ag2O)

Mineralogical studies were made to identical the components of the catalyst, montmorillonite and phylippsite and others were observed. The DTA diagrams indicated characteristic indothermal and exothermal reaction. The adsorption – desorption of N2 measurements were carried out at -196°C. A slight decrease of surface area after having the catalyst been covered with Ag2O.