Cross-Lingual Training of Dense Retrievers for Document Retrieval

Dense retrieval has shown great success for passage ranking in English. However, its effectiveness for non-English languages remains unexplored due to limitation in training resources. In this work, we explore different transfer techniques for document ranking from English annotations to non-English languages. Our experiments reveal that zero-shot model-based transfer using mBERT improves search quality. We find that weakly-supervised target language transfer is competitive compared to generation-based target language transfer, which requires translation models.

Condenser: a Pre-training Architecture for Dense Retrieval

Pre-trained Transformer language models (LM) have become go-to text representation encoders. Prior research fine-tunes deep LMs to encode text sequences such as sentences and passages into single dense vector representations for efficient text comparison and retrieval. However, dense encoders require a lot of data and sophisticated techniques to effectively train and suffer in low data situations. This paper finds a key reason is that standard LMs' internal attention structure is not ready-to-use for dense encoders, which needs to aggregate text information into the dense representation. We propose to pre-train towards dense encoder with a novel Transformer architecture, Condenser, where LM prediction CONditions on DENSE Representation. Our experiments show Condenser improves over standard LM by large margins on various text retrieval and similarity tasks.

In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval

We present an efficient training approach to text retrieval with dense representations that applies knowledge distillation using the ColBERT late-interaction ranking model. Specifically, we propose to transfer the knowledge from a bi-encoder teacher to a student by distilling knowledge from ColBERT's expressive MaxSim operator into a simple dot product. The advantage of the bi-encoder teacher--student setup is that we can efficiently add in-batch negatives during knowledge distillation, enabling richer interactions between teacher and student models. In addition, using ColBERT as the teacher reduces training cost compared to a full cross-encoder. Experiments on the MS MARCO passage and document ranking tasks and data from the TREC 2019 Deep Learning Track demonstrate that our approach helps models learn robust representations for dense retrieval effectively and efficiently.

DeCEMBERT: Learning from Noisy Instructional Videos via Dense Captions and Entropy Minimization

Leveraging large-scale unlabeled web videos such as instructional videos for pre-training followed by task-specific finetuning has become the de facto approach for many video-and-language tasks. However, these instructional videos are very noisy, the accompanying ASR narrations are often incomplete, and can be irrelevant to or temporally misaligned with the visual content, limiting the performance of the models trained on such data. To address these issues, we propose an improved video-and-language pre-training method that first adds automatically-extracted dense region captions from the video frames as auxiliary text input, to provide informative visual cues for learning better video and language associations. Second, to alleviate the temporal misalignment issue, our method incorporates an entropy minimization-based constrained attention loss, to encourage the model to automatically focus on the correct caption from a pool of candidate ASR captions. Our overall approach is named DeCEMBERT (Dense Captions and Entropy Minimization). Comprehensive experiments on three video-and-language tasks (text-to-video retrieval, video captioning, and video question answering) across five datasets demonstrate that our approach outperforms previous state-of-the-art methods. Ablation studies on pre-training and downstream tasks show that adding dense captions and constrained attention loss help improve the model performance. Lastly, we also provide attention visualization to show the effect of applying the proposed constrained attention loss.

RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering

In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers. Typically, the dual-encoder architecture is adopted to learn dense representations of questions and passages for semantic matching. However, it is difficult to effectively train a dual-encoder due to the challenges including the discrepancy between training and inference, the existence of unlabeled positives and limited training data. To address these challenges, we propose an optimized training approach, called RocketQA, to improving dense passage retrieval. We make three major technical contributions in RocketQA, namely cross-batch negatives, denoised hard negatives and data augmentation. The experiment results show that RocketQA significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions. We also conduct extensive experiments to examine the effectiveness of the three strategies in RocketQA. Besides, we demonstrate that the performance of end-to-end QA can be improved based on our RocketQA retriever.

A Theoretical Study of X-ray produced by the Plasma Focus device based on Sing Lee model

The Lee code is applied to characterize the plasma focus in two plasma focus devices UNU/ICTP PFF and Amirkabir plasma focus device (APF), and for optimizing the nitrogen soft x-ray yields based on bank, tubes and operating parameters. It is found that the soft x-ray yield increases with changing pressure until it reaches the maximum value for each plasma focus device, with keeping the bank parameters, operational voltage unchanged but systematically changing other parameters.

dying Landau Damping in dense quantum Plasmas

In This Scientific Paper it had been studied the Effect of Ponderomotive Force on Landau damping of electron Wave in dense quantum Plasmas, by Using quasi quantum kinetic Equation, Using the Corrections due to both of quantum effects and Ponderomotive Force, because of their necessity in Studying Wave Reflection inside Cavity Energy ,and some Physical Properties of Plasmas, then comparing the Results to the other Conclusions in this Field.

Study of Magnetoacoustic solitons in ultra dense quantum electron-positron-ion plasma

In this search, it has been studied the properties of the magnetoacoustic soliton waves in ultra dense quantum plasma and its including ions and electrons and positrons after taking quantum effects of electrons and positrons into consideration due to their Fermionic nature and the quantum diffraction, this is by the quantum Bom potential into two momentum equations of electrons and positrons . It has been studied the solitary waves of small amplitude by using reductive perturbation method. The results have been compared to the solitary waves ones with what others have reached in related references.

Numerical study of soft x-ray yield in two dense plasma focus devices based on Lee code, and comparing it with experimental results using neon filling gas

In this work we carried out some numerical experiments on NX2, UNU/ICTP PFF dense plasma focus device with neon filling gas using Lee code version (RADPFV5.15de.c1) and standard parameters of the devices to compare the value of the soft x-ray yield (Ysxr) emitting from each one. Also we studied the influence some factors on the value of (Ysxr).

Instabilities in Electron Beam in a Dense Warm Inhomogeneous Plasma

We investigate the influence of the variable plasma density on the spatial growth of the beam-plasma instability, considering the model of homogeneous cold beam-inhomogeneous warm plasma system under the condition of the smallness of phase velocity of waves compared to the beam velocity. We determine a direction of the beam with unmagnatized plasma. Considering a one – dimensional electrostatic oscillation when the directions of beam propagation, plasma density gradient and wave electric field coincide with X-axis. To formulate mathematical equation of beam and plasma then we make studying equation linearized, and then study the continuity equation and boundary conditions. Formulating electric field density then study a case in which a plasma is collisional because of its high density and the temperature .Finally we drive absorbent energy and find solutions of these equations then draw it.