In this paper, we propose a new ranking model DR-BERT, which improves the Document Retrieval (DR) task by a task-adaptive training process and a Segmented Token Recovery Mechanism (STRM). In the task-adaptive training, we first pre-train DR-BERT to b
e domain-adaptive and then make the two-phase fine-tuning. In the first-phase fine-tuning, the model learns query-document matching patterns regarding different query types in a pointwise way. Next, in the second-phase fine-tuning, the model learns document-level ranking features and ranks documents with regard to a given query in a listwise manner. Such pointwise plus listwise fine-tuning enables the model to minimize errors in the document ranking by incorporating ranking-specific supervisions. Meanwhile, the model derived from pointwise fine-tuning is also used to reduce noise in the training data of the listwise fine-tuning. On the other hand, we present STRM which can compute OOV word representation and contextualization more precisely in BERT-based models. As an effective strategy in DR-BERT, STRM improves the matching perfromance of OOV words between a query and a document. Notably, our DR-BERT model keeps in the top three on the MS MARCO leaderboard since May 20, 2020.
Copy mechanisms explicitly obtain unchanged tokens from the source (input) sequence to generate the target (output) sequence under the neural seq2seq framework. However, most of the existing copy mechanisms only consider single word copying from the
source sentences, which results in losing essential tokens while copying long spans. In this work, we propose a plug-and-play architecture, namely BioCopy, to alleviate the problem aforementioned. Specifically, in the training stage, we construct a BIO tag for each token and train the original model with BIO tags jointly. In the inference stage, the model will firstly predict the BIO tag at each time step, then conduct different mask strategies based on the predicted BIO label to diminish the scope of the probability distributions over the vocabulary list. Experimental results on two separate generative tasks show that they all outperform the baseline models by adding our BioCopy to the original model structure.
In this work, we conduct a comprehensive investigation on one of the centerpieces of modern machine translation systems: the encoder-decoder attention mechanism. Motivated by the concept of first-order alignments, we extend the (cross-)attention mech
anism by a recurrent connection, allowing direct access to previous attention/alignment decisions. We propose several ways to include such a recurrency into the attention mechanism. Verifying their performance across different translation tasks we conclude that these extensions and dependencies are not beneficial for the translation performance of the Transformer architecture.
Encoder-decoder models have been commonly used for many tasks such as machine translation and response generation. As previous research reported, these models suffer from generating redundant repetition. In this research, we propose a new mechanism f
or encoder-decoder models that estimates the semantic difference of a source sentence before and after being fed into the encoder-decoder model to capture the consistency between two sides. This mechanism helps reduce repeatedly generated tokens for a variety of tasks. Evaluation results on publicly available machine translation and response generation datasets demonstrate the effectiveness of our proposal.
We have a plane mechanical system
consisted of pantograph mechanism and four bar one, with revolute
and sliding joints. Then, we replace each revolute joint with super
elastic hinge. So that, we have a system, strongly recommended, to
achieve the
same goal using minimum energy. The main goal of
this paper is to elaborate a mathematical mechanism able to
estimate the deviations of the considered system before and after
replacing revolute hinges, taking into account the real performance
of the new system through additional large displacements in the
flexural hinges.
In this paper we have a plan mechanical
system consists of two pantograph mechanisms, with revolute and
sliding joints, linked by fixed link . Then, we replace each revolute
joint with super elastic hinge. In this way, we have a system,
strongly recommended, to achieve the same goal using minimum
of energy.
The importance of pantograph mechanism and four Bar one leads
to improve of two mechanisms. As we said in last papers,
decreasing maintenance and having the same goal with low weight
and no friction in nowadays artificial applications, appears as
a
needed need. The experiment refers that using flexural hinges in a
system at least leads to all of that advantages. We have a plan
mechanical system consisted of pantograph mechanism and four
bar one, with revolute and sliding joints. Then, we replace each
revolute joint with super elastic hinge. In this way, we have a gate
to build a system, strongly recommended, to achieve the same goal
using minimum energy. The main goal of this paper is to elaborate
a mathematical mechanism able to estimate the deviations of the
considered system before and after replacing revolute hinges,
taking into account the real performance of the new system through
additional large displacements in the flexural hinges.
This research is based on the study of sciences upon which Mohammed Moftaah's
methodology depends as a source of cognitive and tributary, which is outside the scope
of literary criticism; and that is through the access to the structure of this meth
odology
and analyze its scientific composition. The intended aim of this research is the access to
the most important pillars of the Epistemology of this critic, and the detection of an
important aspect in the critical construction which has not been studied yet. These sciences
are: logic, mathematics, geometry, physics, biology, sociology, and information
technologies.
The methodology of this research is based on the extraction of the practical and the
theoretical science that Moftaah has. and then making it an apparent and theorized
substance , which says That Moftaah has succeeded at some of these sciences such as, ,
biology, Geometry, and some of them is based on formalism such as, mathematics,
information technologies.
We study a planar mechanical system generated by two six-bar
mechanism with revolute joints, then we link them by revolute joint
to become one system . After that we replace each revolute joint
with super elastic hinge. The main purpose of this pa
per is to
elaborate a mathematical method able to estimate the deviations of
the considered system before and after replacing revolute hinges,
taking into account that new system creates large additional
deviations .
The main purpose of this
research is to elaborate a mathematical apparatus able to estimate
the deviations of the considered system before and after replacing
revolute hinges taking into account the real performance of the
novel system through large bending displacements in the flexure
(flexural) hinges.
