Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userTesting Cross-Database Semantic Parsers With Canonical Utterances
اختبار المحللين الدلاليين المعتادين مع الكلمات الكنسية
343
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
ارتفع الأداء القياسي لتحليل الدلالي عبر قاعدة البيانات بشكل مطرد في السنوات الأخيرة، وحفزه باعتماد النماذج اللغوية المدربة مسبقا.ومع ذلك، فقد أظهر العمل الحالي أن المحاورين الدلاليين في قاعدة البيانات عبر الإنترنت يكافحون للتعميم إلى كلمات المستخدمين، وقواعد البيانات وهياكل الاستعلام.للحصول على تفاصيل شفافة حول نقاط القوة والحد من هذه النماذج، نقترح نهج اختبار تشخيصي يعتمد على توليفات التحكم في اللغة الطبيعية الكنسية وأزواج SQL.مستوحاة من قائمة المراجعة، فإننا نميز مجموعة من القدرات الأساسية لنماذج تحليل الدلالية عبر قاعدة البيانات، وتفصيل الطريقة لتوليف بيانات الاختبار المقابلة.قمنا بتقييم مجموعة متنوعة من النماذج عالية الأداء باستخدام النهج المقترح، وحددت العديد من نقاط الضعف غير الواضحة عبر النماذج (E.G. غير قادر على تحديد العديد من الأعمدة بشكل صحيح).يتم إصدار بيانات DataSet و Code كجناح اختبار على http://github.com/hclent/behaviorchecksempar.
The benchmark performance of cross-database semantic parsing has climbed steadily in recent years, catalyzed by the wide adoption of pre-trained language models. Yet existing work have shown that state-of-the-art cross-database semantic parsers struggle to generalize to novel user utterances, databases and query structures. To obtain transparent details on the strengths and limitation of these models, we propose a diagnostic testing approach based on controlled synthesis of canonical natural language and SQL pairs. Inspired by the CheckList, we characterize a set of essential capabilities for cross-database semantic parsing models, and detailed the method for synthesizing the corresponding test data. We evaluated a variety of high performing models using the proposed approach, and identified several non-obvious weaknesses across models (e.g. unable to correctly select many columns). Our dataset and code are released as a test suite at http://github.com/hclent/BehaviorCheckingSemPar.
References used
https://aclanthology.org/
rate research
Read More
We explore the use of large pretrained language models as few-shot semantic parsers. The goal in semantic parsing is to generate a structured meaning representation given a natural language input. However, language models are trained to generate natu
ral language. To bridge the gap, we use language models to paraphrase inputs into a controlled sublanguage resembling English that can be automatically mapped to a target meaning representation. Our results demonstrate that with only a small amount of data and very little code to convert into English-like representations, our blueprint for rapidly bootstrapping semantic parsers leads to surprisingly effective performance on multiple community tasks, greatly exceeding baseline methods also trained on the same limited data.
Deep Learning-based NLP systems can be sensitive to unseen tokens and hard to learn with high-dimensional inputs, which critically hinder learning generalization. We introduce an approach by grouping input words based on their semantic diversity to s
implify input language representation with low ambiguity. Since the semantically diverse words reside in different contexts, we are able to substitute words with their groups and still distinguish word meanings relying on their contexts. We design several algorithms that compute diverse groupings based on random sampling, geometric distances, and entropy maximization, and we prove formal guarantees for the entropy-based algorithms. Experimental results show that our methods generalize NLP models and demonstrate enhanced accuracy on POS tagging and LM tasks and significant improvements on medium-scale machine translation tasks, up to +6.5 BLEU points. Our source code is available at https://github.com/abdulrafae/dg.
While cross-lingual techniques are finding increasing success in a wide range of Natural Language Processing tasks, their application to Semantic Role Labeling (SRL) has been strongly limited by the fact that each language adopts its own linguistic f
ormalism, from PropBank for English to AnCora for Spanish and PDT-Vallex for Czech, inter alia. In this work, we address this issue and present a unified model to perform cross-lingual SRL over heterogeneous linguistic resources. Our model implicitly learns a high-quality mapping for different formalisms across diverse languages without resorting to word alignment and/or translation techniques. We find that, not only is our cross-lingual system competitive with the current state of the art but that it is also robust to low-data scenarios. Most interestingly, our unified model is able to annotate a sentence in a single forward pass with all the inventories it was trained with, providing a tool for the analysis and comparison of linguistic theories across different languages. We release our code and model at https://github.com/SapienzaNLP/unify-srl.
Despite their success, modern language models are fragile. Even small changes in their training pipeline can lead to unexpected results. We study this phenomenon by examining the robustness of ALBERT (Lan et al., 2020) in combination with Stochastic
Weight Averaging (SWA)---a cheap way of ensembling---on a sentiment analysis task (SST-2). In particular, we analyze SWA's stability via CheckList criteria (Ribeiro et al., 2020), examining the agreement on errors made by models differing only in their random seed. We hypothesize that SWA is more stable because it ensembles model snapshots taken along the gradient descent trajectory. We quantify stability by comparing the models' mistakes with Fleiss' Kappa (Fleiss, 1971) and overlap ratio scores. We find that SWA reduces error rates in general; yet the models still suffer from their own distinct biases (according to CheckList).
This paper aims to test weak form efficiency in
Damascus , Amman , Muscat securities market .It examines daily
stock return index during ( 1 - 4- 2010 ) , ( 31 - 12 - 2016 ) using
normal distribution test , runs test , autocorrelation test , unit
root
test , variance ratio test , auto regressive integrated moving average
test
الصيغة الضعيفة للكفاءة
اختبار التوزيع الطبيعي
اختبار التكرارات
اختبار الارتباط التسلسلي
اختبار نسبة التباين
اختبار الانحدار الذاتي و المتوسط المتحرك المتكامل
weak form efficiency
normal distribution test
Runs test
Autocorrelation test
Variance ratio test
auto regressive integrated moving average test
المزيد..
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
