Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userA Deep Decomposable Model for Disentangling Syntax and Semantics in Sentence Representation
نموذج متحلل عميق ل Secornangling بناء الجملة والدلالات في تمثيل الجملة
357
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
في الآونة الأخيرة، تقدم DEVENTANGLEMEMEMENEM بناء على شبكة خدرية توليدية أو AutoNCoder التباين بشكل كبير أداء التطبيقات المتنوعة في مجالات السيرة الذاتية و NLP.ومع ذلك، لا تزال هذه النماذج تعمل على مستويات خشنة في تحسين الخصائص ذات الصلة ارتباطا وثيقا، مثل بناء الجملة والدلالات باللغات البشرية.تقدم هذه الورقة نموذجا متحللا عميقا يستند إلى بناء جملة VAE ل DisentAnge و DeMantics باستخدام عقوبات الارتباط الكلية على اختلافات KL.والجدير بالذكر أننا نتحلل مدة الاختلاف KL من VAE الأصلي بحيث يمكن فصل المتغيرات الكامنة التي تم إنشاؤها بطريقة أكثر وضوحا وتفسيرا.تبين التجارب على مجموعات البيانات القياسية أن نموذجنا المقترح يمكن أن يحسن بشكل كبير من جودة الإعانات بين التمثيلات النحوية والدلية لمهام التشابه الدلالي ومهام التشابه النحوية.
Recently, disentanglement based on a generative adversarial network or a variational autoencoder has significantly advanced the performance of diverse applications in CV and NLP domains. Nevertheless, those models still work on coarse levels in the disentanglement of closely related properties, such as syntax and semantics in human languages. This paper introduces a deep decomposable model based on VAE to disentangle syntax and semantics by using total correlation penalties on KL divergences. Notably, we decompose the KL divergence term of the original VAE so that the generated latent variables can be separated in a more clear-cut and interpretable way. Experiments on benchmark datasets show that our proposed model can significantly improve the disentanglement quality between syntactic and semantic representations for semantic similarity tasks and syntactic similarity tasks.
References used
https://aclanthology.org/
rate research
Read More
Pre-trained language models have achieved huge success on a wide range of NLP tasks. However, contextual representations from pre-trained models contain entangled semantic and syntactic information, and therefore cannot be directly used to derive use
ful semantic sentence embeddings for some tasks. Paraphrase pairs offer an effective way of learning the distinction between semantics and syntax, as they naturally share semantics and often vary in syntax. In this work, we present ParaBART, a semantic sentence embedding model that learns to disentangle semantics and syntax in sentence embeddings obtained by pre-trained language models. ParaBART is trained to perform syntax-guided paraphrasing, based on a source sentence that shares semantics with the target paraphrase, and a parse tree that specifies the target syntax. In this way, ParaBART learns disentangled semantic and syntactic representations from their respective inputs with separate encoders. Experiments in English show that ParaBART outperforms state-of-the-art sentence embedding models on unsupervised semantic similarity tasks. Additionally, we show that our approach can effectively remove syntactic information from semantic sentence embeddings, leading to better robustness against syntactic variation on downstream semantic tasks.
External syntactic and semantic information has been largely ignored by existing neural coreference resolution models. In this paper, we present a heterogeneous graph-based model to incorporate syntactic and semantic structures of sentences. The prop
osed graph contains a syntactic sub-graph where tokens are connected based on a dependency tree, and a semantic sub-graph that contains arguments and predicates as nodes and semantic role labels as edges. By applying a graph attention network, we can obtain syntactically and semantically augmented word representation, which can be integrated using an attentive integration layer and gating mechanism. Experiments on the OntoNotes 5.0 benchmark show the effectiveness of our proposed model.
Existing text style transfer (TST) methods rely on style classifiers to disentangle the text's content and style attributes for text style transfer. While the style classifier plays a critical role in existing TST methods, there is no known investiga
tion on its effect on the TST methods. In this paper, we conduct an empirical study on the limitations of the style classifiers used in existing TST methods. We demonstrated that the existing style classifiers cannot learn sentence syntax effectively and ultimately worsen existing TST models' performance. To address this issue, we propose a novel Syntax-Aware Controllable Generation (SACG) model, which includes a syntax-aware style classifier that ensures learned style latent representations effectively capture the sentence structure for TST. Through extensive experiments on two popular text style transfer tasks, we show that our proposed method significantly outperforms twelve state-of-the-art methods. Our case studies have also demonstrated SACG's ability to generate fluent target-style sentences that preserved the original content.
Sentence weighting is a simple and powerful domain adaptation technique. We carry out domain classification for computing sentence weights with 1) language model cross entropy difference 2) a convolutional neural network 3) a Recursive Neural Tensor
Network. We compare these approaches with regard to domain classification accuracy and and study the posterior probability distributions. Then we carry out NMT experiments in the scenario where we have no in-domain parallel corpora and and only very limited in-domain monolingual corpora. Here and we use the domain classifier to reweight the sentences of our out-of-domain training corpus. This leads to improvements of up to 2.1 BLEU for German to English translation.
This research shows the concept of sentence syntax and the text
syntax and the difference between them, beside their respective
areas .It also tries to specify the obstacles which prevent the
progress of this kind of linguistic lesson in our Arabi
an collages
.Then it stops at the trends of linguistic studies where such kind of
linguistic lesson appears .Also tries to monitor the reality of this
lingual lesson in the Syrian collages through one sample ,that is Al
Baath University .Finally finishes by the most important
recommendations which can contribute in developing this kind of
lingual lesson .
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
