Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userCross-Lingual Word Embedding Refinement by $\ell_1$ Norm Optimisation
كلمة تبادل الكلمة تضمين التوصيل حسب التحسين $ \ ELL_1 $
299
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تزيين الكلمات المتبقية كلمة (CLWES) ترميز الكلمات من لغتين أو أكثر في مساحة مشتركة عالية الأبعاد التي تمثل ناقلات تمثل الكلمات ذات معنى مماثل (بغض النظر عن اللغة) عن كثب. تعلم الأساليب الحالية لبناء تعيينات CLWES عالية الجودة التي تقلل من وظيفة خسارة المعايير ℓ2. ومع ذلك، فقد ثبت أن هدف التحسين هذا هو حساسا للقيم المتطرفة. بناء على قاعدة مانهاتن الأكثر قوة (AKA. ℓ1 NORM) معيار الجيدة، تقترح هذه الورقة خطوة بسيطة بعد المعالجة لتحسين CLWES. ميزة هذا النهج هي أنه غير ملائم تماما للعملية التدريبية للفقر الأصلي ويمكن تطبيقها على نطاق واسع. يتم إجراء تجارب واسعة النطاق التي تنطوي على عشرة لغات متنوعة وموظفة مدربة على شريعة مختلفة. تظهر نتائج التقييم المستندة إلى تحريض المعجم الثنائي اللغة والتحويل عبر اللغات لمهام الاستدلال باللغة الطبيعية أن الصقل ℓ1 يتفوق بشكل كبير على أربعة خطوط خطوط خطوط خطوط أخرى في كل من الإعدادات الإشرافية غير المشرف. لذلك يوصى باعتماد هذه الاستراتيجية كمعيار لأساليب CLWE.
Cross-Lingual Word Embeddings (CLWEs) encode words from two or more languages in a shared high-dimensional space in which vectors representing words with similar meaning (regardless of language) are closely located. Existing methods for building high-quality CLWEs learn mappings that minimise the ℓ2 norm loss function. However, this optimisation objective has been demonstrated to be sensitive to outliers. Based on the more robust Manhattan norm (aka. ℓ1 norm) goodness-of-fit criterion, this paper proposes a simple post-processing step to improve CLWEs. An advantage of this approach is that it is fully agnostic to the training process of the original CLWEs and can therefore be applied widely. Extensive experiments are performed involving ten diverse languages and embeddings trained on different corpora. Evaluation results based on bilingual lexicon induction and cross-lingual transfer for natural language inference tasks show that the ℓ1 refinement substantially outperforms four state-of-the-art baselines in both supervised and unsupervised settings. It is therefore recommended that this strategy be adopted as a standard for CLWE methods.
References used
https://aclanthology.org/
rate research
Read More
Word embedding is essential for neural network models for various natural language processing tasks. Since the word embedding usually has a considerable size, in order to deploy a neural network model having it on edge devices, it should be effective
ly compressed. There was a study for proposing a block-wise low-rank approximation method for word embedding, called GroupReduce. Even if their structure is effective, the properties behind the concept of the block-wise word embedding compression were not sufficiently explored. Motivated by this, we improve GroupReduce in terms of word weighting and structuring. For word weighting, we propose a simple yet effective method inspired by the term frequency-inverse document frequency method and a novel differentiable method. Based on them, we construct a discriminative word embedding compression algorithm. In the experiments, we demonstrate that the proposed algorithm more effectively finds word weights than competitors in most cases. In addition, we show that the proposed algorithm can act like a framework through successful cooperation with quantization.
Word embeddings are powerful representations that form the foundation of many natural language processing architectures, both in English and in other languages. To gain further insight into word embeddings, we explore their stability (e.g., overlap b
etween the nearest neighbors of a word in different embedding spaces) in diverse languages. We discuss linguistic properties that are related to stability, drawing out insights about correlations with affixing, language gender systems, and other features. This has implications for embedding use, particularly in research that uses them to study language trends.
Sarcasm detection is of great importance in understanding people's true sentiments and opinions. Many online feedbacks, reviews, social media comments, etc. are sarcastic. Several researches have already been done in this field, but most researchers
studied the English sarcasm analysis compared to the researches are done in Arabic sarcasm analysis because of the Arabic language challenges. In this paper, we propose a new approach for improving Arabic sarcasm detection. Our approach is using data augmentation, contextual word embedding and random forests model to get the best results. Our accuracy in the shared task on sarcasm and sentiment detection in Arabic was 0.5189 for F1-sarcastic as the official metric using the shared dataset ArSarcasmV2 (Abu Farha, et al., 2021).
In this work, we analyze the performance and properties of cross-lingual word embedding models created by mapping-based alignment methods. We use several measures of corpus and embedding similarity to predict BLI scores of cross-lingual embedding map
pings over three types of corpora, three embedding methods and 55 language pairs. Our experimental results corroborate that instead of mere size, the amount of common content in the training corpora is essential. This phenomenon manifests in that i) despite of the smaller corpus sizes, using only the comparable parts of Wikipedia for training the monolingual embedding spaces to be mapped is often more efficient than relying on all the contents of Wikipedia, ii) the smaller, in return less diversified Spanish Wikipedia works almost always much better as a training corpus for bilingual mappings than the ubiquitously used English Wikipedia.
Adapting word order from one language to another is a key problem in cross-lingual structured prediction. Current sentence encoders (e.g., RNN, Transformer with position embeddings) are usually word order sensitive. Even with uniform word form repres
entations (MUSE, mBERT), word order discrepancies may hurt the adaptation of models. In this paper, we build structured prediction models with bag-of-words inputs, and introduce a new reordering module to organizing words following the source language order, which learns task-specific reordering strategies from a general-purpose order predictor model. Experiments on zero-shot cross-lingual dependency parsing, POS tagging, and morphological tagging show that our model can significantly improve target language performances, especially for languages that are distant from the source language.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
