Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userA Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space
تحليل متعدد اللغات بشكل كبير من تبادل اللغة في الفضاء التضمين المشترك
309
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
في طرازات اللغة عبر اللغات، تعيش تمثيلات للعديد من اللغات المختلفة في نفس المساحة. هنا، نحقق في العوامل اللغوية وغير اللغوية التي تؤثر على محاذاة على مستوى الجملة في نماذج اللغة المحددة مسبقا بين 101 لغة و 5،050 زوج لغة. باستخدام LASTE BERT-القائم على BERT و LASER المستندة إلى BILSTM كنماذجنا، والكتاب المقدس كجورتنا، نحسب مقياسا يستند إلى المهمة لمحاذاة عبر اللغات في شكل أداء استرجاع BiteXT، بالإضافة إلى أربعة تدابير جوهرية لمساحة المتجهات المحاذاة والتزييف. ثم ندرس مجموعة من الميزات اللغوية واللغوية واللغوية والمتعلقة التدريبية كتنبؤ محتمل من مقاييس المحاذاة. تظهر نتائج تحليلاتنا أن اتفاقية ترتيب الكلمات والاتفاق في التعقيد المورفولوجي هي اثنتان من أقوى المتنبئ اللغوي للقلق. نلاحظ أيضا البيانات التدريبية في الأسرة كمؤشر أقوى من بيانات التدريب المحددة باللغة في جميع المجالات. نحن نتحقق من بعض النتائج اللغوية لدينا من خلال النظر في تأثير تجزئة مورفولوجية على محاذاة اللغة الإنجليزية - غير الأجنبية، بالإضافة إلى دراسة تأثير اتفاقية ترتيب الكلمات على ISomorphism ل 66 أزواج لغة الطلقة الصفرية من كائن مختلف. نحن نجعل البيانات والرمز تجاربنا متاحة للجمهور.
In cross-lingual language models, representations for many different languages live in the same space. Here, we investigate the linguistic and non-linguistic factors affecting sentence-level alignment in cross-lingual pretrained language models for 101 languages and 5,050 language pairs. Using BERT-based LaBSE and BiLSTM-based LASER as our models, and the Bible as our corpus, we compute a task-based measure of cross-lingual alignment in the form of bitext retrieval performance, as well as four intrinsic measures of vector space alignment and isomorphism. We then examine a range of linguistic, quasi-linguistic, and training-related features as potential predictors of these alignment metrics. The results of our analyses show that word order agreement and agreement in morphological complexity are two of the strongest linguistic predictors of cross-linguality. We also note in-family training data as a stronger predictor than language-specific training data across the board. We verify some of our linguistic findings by looking at the effect of morphological segmentation on English-Inuktitut alignment, in addition to examining the effect of word order agreement on isomorphism for 66 zero-shot language pairs from a different corpus. We make the data and code for our experiments publicly available.
References used
https://aclanthology.org/
rate research
Read More
For children, the system trained on a large corpus of adult speakers performed worse than a system trained on a much smaller corpus of children's speech. This is due to the acoustic mismatch between training and testing data. To capture more acoustic
variability we trained a shared system with mixed data from adults and children. The shared system yields the best EER for children with no degradation for adults. Thus, the single system trained with mixed data is applicable for speaker verification for both adults and children.
The recent Text-to-Text Transfer Transformer'' (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce mT5, a multilingual variant of T5 th
at was pre-trained on a new Common Crawl-based dataset covering 101 languages. We detail the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual benchmarks. We also describe a simple technique to prevent accidental translation'' in the zero-shot setting, where a generative model chooses to (partially) translate its prediction into the wrong language. All of the code and model checkpoints used in this work are publicly available.
Lexical normalization is the task of transforming an utterance into its standardized form. This task is beneficial for downstream analysis, as it provides a way to harmonize (often spontaneous) linguistic variation. Such variation is typical for soci
al media on which information is shared in a multitude of ways, including diverse languages and code-switching. Since the seminal work of Han and Baldwin (2011) a decade ago, lexical normalization has attracted attention in English and multiple other languages. However, there exists a lack of a common benchmark for comparison of systems across languages with a homogeneous data and evaluation setup. The MultiLexNorm shared task sets out to fill this gap. We provide the largest publicly available multilingual lexical normalization benchmark including 13 language variants. We propose a homogenized evaluation setup with both intrinsic and extrinsic evaluation. As extrinsic evaluation, we use dependency parsing and part-of-speech tagging with adapted evaluation metrics (a-LAS, a-UAS, and a-POS) to account for alignment discrepancies. The shared task hosted at W-NUT 2021 attracted 9 participants and 18 submissions. The results show that neural normalization systems outperform the previous state-of-the-art system by a large margin. Downstream parsing and part-of-speech tagging performance is positively affected but to varying degrees, with improvements of up to 1.72 a-LAS, 0.85 a-UAS, and 1.54 a-POS for the winning system.
We present the first annotated corpus for multilingual analysis of potentially unfair clauses in online Terms of Service. The data set comprises a total of 100 contracts, obtained from 25 documents annotated in four different languages: English, Germ
an, Italian, and Polish. For each contract, potentially unfair clauses for the consumer are annotated, for nine different unfairness categories. We show how a simple yet efficient annotation projection technique based on sentence embeddings could be used to automatically transfer annotations across languages.
Abstract Prior studies in multilingual language modeling (e.g., Cotterell et al., 2018; Mielke et al., 2019) disagree on whether or not inflectional morphology makes languages harder to model. We attempt to resolve the disagreement and extend those s
tudies. We compile a larger corpus of 145 Bible translations in 92 languages and a larger number of typological features.1 We fill in missing typological data for several languages and consider corpus-based measures of morphological complexity in addition to expert-produced typological features. We find that several morphological measures are significantly associated with higher surprisal when LSTM models are trained with BPE-segmented data. We also investigate linguistically motivated subword segmentation strategies like Morfessor and Finite-State Transducers (FSTs) and find that these segmentation strategies yield better performance and reduce the impact of a language's morphology on language modeling.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
