Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userA Non-Linear Structural Probe
التحقيق الهيكلية غير الخطية
178
0 0
0.0
(
0
)
Created by
Shamra Editor
Ask ChatGPT about the research
التحقيقات هي النماذج المبينة للتحقيق في ترميز المعرفة --- E.G. هيكل النحوي --- في تمثيلات السياقية. غالبا ما يتم تصميم تحقيقات البساطة، مما أدى إلى قيود على تصميم التحقيق الذي قد لا يسمح بالاستغلال الكامل لهيكل المعلومات المشفرة؛ واحد من هذا القيادة هو الخطي. ندرس حالة التحقيق الهيكلي (Hewitt and Manning، 2019)، والتي تهدف إلى التحقيق في ترميز الهيكل النحوي في تمثيلات سياقية من خلال تعلم التحولات الخطية فقط. من خلال مراعاة أن التحقيق الهيكلي يتعلم متريا، يمكننا أن نكون قادرين على تحليلها وتطوير متغير غير خطي رواية مع عدد متطابق من المعلمات. نحن نختبر في 6 لغات وتجد أن نواة الوظيفة الرئيسية (RBF)، بالتزامن مع التنظيم، وتحقق تحسنا كبيرا إحصائيا على أساس الأساس بجميع اللغات --- يعني أن جزءا على الأقل من المعرفة النحوية يتم تشفيره خطيا. نستنتج من خلال مناقشة كيفية تشبه Kernel RBF طبقات الانتباه ذاتية بيرت ومكهن أن هذه التشابه يؤدي إلى أداء التحقيق القائم على RBF.
Probes are models devised to investigate the encoding of knowledge---e.g. syntactic structure---in contextual representations. Probes are often designed for simplicity, which has led to restrictions on probe design that may not allow for the full exploitation of the structure of encoded information; one such restriction is linearity. We examine the case of a structural probe (Hewitt and Manning, 2019), which aims to investigate the encoding of syntactic structure in contextual representations through learning only linear transformations. By observing that the structural probe learns a metric, we are able to kernelize it and develop a novel non-linear variant with an identical number of parameters. We test on 6 languages and find that the radial-basis function (RBF) kernel, in conjunction with regularization, achieves a statistically significant improvement over the baseline in all languages---implying that at least part of the syntactic knowledge is encoded non-linearly. We conclude by discussing how the RBF kernel resembles BERT's self-attention layers and speculate that this resemblance leads to the RBF-based probe's stronger performance.
References used
https://aclanthology.org/
rate research
Read More
In this paper we consider the properties of linear systems by means of
directed graphs and numerical structures. We also state efficient algorithms
for determining an approximate number of the non-zero terms within
determinants' expressions of the
ir matrices. The stated algorithms make use of
trees representing numerical structures which contains the indices of the nonzero
terms.
This paper yields interesting results used in practical engineering
applications which include linear systems with sparse matrices, for example:
networks, electronic circuits, earth velocities boxes (gearboxes), multi-works
systems ...etc.
Unsupervised cross-domain dependency parsing is to accomplish domain adaptation for dependency parsing without using labeled data in target domain. Existing methods are often of the pseudo-annotation type, which generates data through self-annotation
of the base model and performing iterative training. However, these methods fail to consider the change of model structure for domain adaptation. In addition, the structural information contained in the text cannot be fully exploited. To remedy these drawbacks, we propose a Semantics-Structure Adaptative Dependency Parser (SSADP), which accomplishes unsupervised cross-domain dependency parsing without relying on pseudo-annotation or data selection. In particular, we design two feature extractors to extract semantic and structural features respectively. For each type of features, a corresponding feature adaptation method is utilized to achieve domain adaptation to align the domain distribution, which effectively enhances the unsupervised cross-domain transfer capability of the model. We validate the effectiveness of our model by conducting experiments on the CODT1 and CTB9 respectively, and the results demonstrate that our model can achieve consistent performance improvement. Besides, we verify the structure transfer ability of the proposed model by introducing Weisfeiler-Lehman Test.
In this paper, spline collocation method is considered for solving two forms of problems. The first form is general linear sixth-order boundary-value problem (BVP), and the second form is nonlinear sixth-order initial value problem (IVP). The existen
ce, uniqueness, error estimation and convergence analysis of purpose methods are investigated. The study shows that proposed spline method with three collocation points can find the spline solutions and their derivatives up to sixth-order of the two BVP and IVP, thus is very effective tools in numerically solving such problems. Several examples are given to verify the reliability and efficiency of the proposed method. Comparisons are made to reconfirm the efficiency and accuracy of the suggested techniques.
In this work , The fifth order non-polynomial spline functions is
used to solve linear volterra integral equations with weakly
singular kernel .
Numerical examples are presented to illustrate the applications
of this method and to compare the computed results with
other numerical methods.
Web search is an essential way for humans to obtain information, but it's still a great challenge for machines to understand the contents of web pages. In this paper, we introduce the task of web-based structural reading comprehension. Given a web pa
ge and a question about it, the task is to find an answer from the web page. This task requires a system not only to understand the semantics of texts but also the structure of the web page. Moreover, we proposed WebSRC, a novel Web-based Structural Reading Comprehension dataset. WebSRC consists of 400K question-answer pairs, which are collected from 6.4K web pages with corresponding HTML source code, screenshots, and metadata. Each question in WebSRC requires a certain structural understanding of a web page to answer, and the answer is either a text span on the web page or yes/no. We evaluate various strong baselines on our dataset to show the difficulty of our task. We also investigate the usefulness of structural information and visual features. Our dataset and baselines have been publicly available.
suggested questions
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
