Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userEvaluation Datasets for Cross-lingual Semantic Textual Similarity
مجموعات بيانات التقييم التشابه الدلالي عبر اللغات
407
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
تقدير أنظمة التشابه الدلالي النصي (STS) درجة تشابه معنى بين جملتين.تقدر أنظمة STS عبر اللغات درجة تشابه معنى بين جملتين، كل منها بلغة مختلفة.عادة ما تستخدم الخوارزميات الحديثة عادة نهجا بالغضب بشدة، يصعب استخدامه لغات ضعف الموارد.ومع ذلك، يحتاج أي نهج للحصول على بيانات التقييم لتأكيد النتائج.من أجل تبسيط عملية التقييم لغات ضعف الموارد (من حيث مجموعات بيانات تقييم STS)، نقدم مجموعات بيانات جديدة ل STS عبر اللغات والأحمر غير المباشر لغات دون بيانات التقييم هذه.نقدم أيضا نتائج العديد من الطرق الحديثة على هذه البيانات التي يمكن استخدامها كأساس للحصول على مزيد من البحث.نعتقد أن هذه المقالة لن تمد فقط أبحاث STS الحالية فقط إلى لغات أخرى، ولكنها ستشجع أيضا المنافسة على هذه بيانات التقييم الجديدة.
Semantic textual similarity (STS) systems estimate the degree of the meaning similarity between two sentences. Cross-lingual STS systems estimate the degree of the meaning similarity between two sentences, each in a different language. State-of-the-art algorithms usually employ a strongly supervised, resource-rich approach difficult to use for poorly-resourced languages. However, any approach needs to have evaluation data to confirm the results. In order to simplify the evaluation process for poorly-resourced languages (in terms of STS evaluation datasets), we present new datasets for cross-lingual and monolingual STS for languages without this evaluation data. We also present the results of several state-of-the-art methods on these data which can be used as a baseline for further research. We believe that this article will not only extend the current STS research to other languages, but will also encourage competition on this new evaluation data.
References used
https://aclanthology.org/
rate research
Read More
ROUGE is a widely used evaluation metric in text summarization. However, it is not suitable for the evaluation of abstractive summarization systems as it relies on lexical overlap between the gold standard and the generated summaries. This limitation
becomes more apparent for agglutinative languages with very large vocabularies and high type/token ratios. In this paper, we present semantic similarity models for Turkish and apply them as evaluation metrics for an abstractive summarization task. To achieve this, we translated the English STSb dataset into Turkish and presented the first semantic textual similarity dataset for Turkish as well. We showed that our best similarity models have better alignment with average human judgments compared to ROUGE in both Pearson and Spearman correlations.
We propose a method to distill a language-agnostic meaning embedding from a multilingual sentence encoder. By removing language-specific information from the original embedding, we retrieve an embedding that fully represents the sentence's meaning. T
he proposed method relies only on parallel corpora without any human annotations. Our meaning embedding allows efficient cross-lingual sentence similarity estimation by simple cosine similarity calculation. Experimental results on both quality estimation of machine translation and cross-lingual semantic textual similarity tasks reveal that our method consistently outperforms the strong baselines using the original multilingual embedding. Our method consistently improves the performance of any pre-trained multilingual sentence encoder, even in low-resource language pairs where only tens of thousands of parallel sentence pairs are available.
While cross-lingual techniques are finding increasing success in a wide range of Natural Language Processing tasks, their application to Semantic Role Labeling (SRL) has been strongly limited by the fact that each language adopts its own linguistic f
ormalism, from PropBank for English to AnCora for Spanish and PDT-Vallex for Czech, inter alia. In this work, we address this issue and present a unified model to perform cross-lingual SRL over heterogeneous linguistic resources. Our model implicitly learns a high-quality mapping for different formalisms across diverse languages without resorting to word alignment and/or translation techniques. We find that, not only is our cross-lingual system competitive with the current state of the art but that it is also robust to low-data scenarios. Most interestingly, our unified model is able to annotate a sentence in a single forward pass with all the inventories it was trained with, providing a tool for the analysis and comparison of linguistic theories across different languages. We release our code and model at https://github.com/SapienzaNLP/unify-srl.
Lack of training data presents a grand challenge to scaling out spoken language understanding (SLU) to low-resource languages. Although various data augmentation approaches have been proposed to synthesize training data in low-resource target languag
es, the augmented data sets are often noisy, and thus impede the performance of SLU models. In this paper we focus on mitigating noise in augmented data. We develop a denoising training approach. Multiple models are trained with data produced by various augmented methods. Those models provide supervision signals to each other. The experimental results show that our method outperforms the existing state of the art by 3.05 and 4.24 percentage points on two benchmark datasets, respectively. The code will be made open sourced on github.
Text Similarity is an important task in several application fields, such as information retrieval, plagiarism detection, machine translation, topic detection, text classification, text summarization and others. Finding similarity between two texts, p
aragraphs or sentences, is based on measuring, directly or indirectly, the similarity between words.
There are two known types of words similarity: lexical and semantic.
The first one handles the words as a stream of characters: words are similar lexically if they share the same characters in the same order.
The second type aims to quantify the degree to which two words are semantically related. As an example they can be, synonyms, represent the same thing or they are used in the same context.
In this article we focus our investigation on measuring the semantic similarity between Arabic sentences using several representations
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
