Knowledge Distillation (KD) is extensively used in Natural Language Processing to compress the pre-training and task-specific fine-tuning phases of large neural language models. A student model is trained to minimize a convex combination of the predi
ction loss over the labels and another over the teacher output. However, most existing works either fix the interpolating weight between the two losses apriori or vary the weight using heuristics. In this work, we propose a novel sample-wise loss weighting method, RW-KD. A meta-learner, simultaneously trained with the student, adaptively re-weights the two losses for each sample. We demonstrate, on 7 datasets of the GLUE benchmark, that RW-KD outperforms other loss re-weighting methods for KD.
Transformer-based models have gained increasing popularity achieving state-of-the-art performance in many research fields including speech translation. However, Transformer's quadratic complexity with respect to the input sequence length prevents its
adoption as is with audio signals, which are typically represented by long sequences. Current solutions resort to an initial sub-optimal compression based on a fixed sampling of raw audio features. Therefore, potentially useful linguistic information is not accessible to higher-level layers in the architecture. To solve this issue, we propose Speechformer, an architecture that, thanks to reduced memory usage in the attention layers, avoids the initial lossy compression and aggregates information only at a higher level according to more informed linguistic criteria. Experiments on three language pairs (en→de/es/nl) show the efficacy of our solution, with gains of up to 0.8 BLEU on the standard MuST-C corpus and of up to 4.0 BLEU in a low resource scenario.
Fine-grained classification involves dealing with datasets with larger number of classes with subtle differences between them. Guiding the model to focus on differentiating dimensions between these commonly confusable classes is key to improving perf
ormance on fine-grained tasks. In this work, we analyse the contrastive fine-tuning of pre-trained language models on two fine-grained text classification tasks, emotion classification and sentiment analysis. We adaptively embed class relationships into a contrastive objective function to help differently weigh the positives and negatives, and in particular, weighting closely confusable negatives more than less similar negative examples. We find that Label-aware Contrastive Loss outperforms previous contrastive methods, in the presence of larger number and/or more confusable classes, and helps models to produce output distributions that are more differentiated.
The complexity loss paradox, which posits that individuals suffering from disease exhibit surprisingly predictable behavioral dynamics, has been observed in a variety of both human and animal physiological systems. The recent advent of online text-ba
sed therapy presents a new opportunity to analyze the complexity loss paradox in a novel operationalization: linguistic complexity loss in text-based therapy conversations. In this paper, we analyze linguistic complexity correlates of mental health in the online therapy messages sent between therapists and 7,170 clients who provided 30,437 corresponding survey responses on their anxiety. We found that when clients reported more anxiety, they showed reduced lexical diversity as estimated by the moving average type-token ratio. Therapists, on the other hand, used language of higher reading difficulty, syntactic complexity, and age of acquisition when clients were more anxious. Finally, we found that clients, and to an even greater extent, therapists, exhibited consistent levels of many linguistic complexity measures. These results demonstrate how linguistic analysis of text-based communication can be leveraged as a marker for anxiety, an exciting prospect in a time of both increased online communication and increased mental health issues.
The aim of this research is to predict the quantities of soil lost by the water erosion in the
Al-Hawiz Dam basin area using GIS and RUSL. R factor was calculated through
matimatical equation after collecting rain data during 2008-2017 from weather
station at
Basel-Al-Assad airport .k value of each soil sampl was calculated after determination of
txture,structure,saturated hydrolic conductivity, and organic matter).a map were prepared
showed local distribution of k values .slop factor was determined as well as using DEM for
studied region, and slop map was introduced in mathematical equation through a GIS to
obtain LS map .NDV used for studied region to calculate C map.To obtain predictive map
of soil lost quantitis ,maps of LS,C,K was multiplicated with R value.
The results showed that R value in studied region 342.78 ,while k factor value was
0.7-0.28.soil with low value concentrated at medium part of studied region,whil slop
factor value was between 0 and 38.87.C factor value was 0.29 at west part and 0.98 at east
part .prediction map of lost quantites was classified in to 4 degrees according erosion risk (
very low risk ,low,medium,high .The results of soil lost quantities were classified in to 4
classes in studied region : very low( 0-5) t/h/year,low( 5-12 ) t/h/year and medium ( 12-24
t/h/year and severe in which soil loss exceeded 24 t/h/year
The idea of this research is about the study of the change of thermal
conductivity of blend mixture composed of Syrian woolen fibers with concrete
of equal weight percentages of 0,5%, 1%, 2% and 4% respectively. the purpose
is to diminish the ratio of the thermal loss in the building.
This investigation was carried out in the fields and laboratories of Pome and
Grapevine Department, GCSAR/Syria, on some apple varieties i.e. Richared,
Mutsu and Tourly Winsap to determine the optimal harvest date and its effect on
storability. Fr
uits were harvested in (16/9, 26/9, 7/10, 17/10 and 30/10) depending
on physical and chemical maturity indicators, and stored with the study of changing
in quality indicators and weight lost monthly. The results showed that the studied
varieties differed in their storability according to harvest date, Richared fruits in
26/9 showed long storage for seven months with weight loss of 4.9%, fruit firmness
6.4 kg/cm2, and 18.7% TSS. While the storability was six months for Mutsu fruits
which harvested in 7/10 with weight loss of 4.4%, and fruit firmness 6.1 kg/cm2,
and finally, Tourly Winsap fruits which were stored for five months in 7 and 17/10
harvest dates with fruit firmness 5.1 kg/cm2. Moreover, the three varieties were able
to storage for three months when harvested in 30/10 with weight loss of 3.3%, 4.4%
and 4.5% in Tourly Winsap, Richared and Mutsu respectively, which considered
as consuming maturity for these varieties. These results indicated to the various
storage periods according to fruits harvest time, and the storability, which assists
in regulating offer and demand, and providing markets with fruits for long period,
however harvesting at optimal time reducing the total weight loss, in addition to
high quality indicators.
In this paper, we assess the Voice Over Internet Protocol
performance by comparing the performance of two protocols
used in VOIP such as SIP and H.323. Moreover, we evaluate
the quality indicators such as delay and packets loss. For this
purpose
OPNET simulator is used as suitable simulation
technology.
This research aims to study the reduction of air heaters corrosion in
steam boilers and increasing their efficiency , by the application of
combined heating of air by steam and exhaust gases.
The experimental data of this research demonstrates tha
theater
corrosion could bereduced by maintaining the exhaust gases
temperature above the dew point of sulfuric acid depending onthe
amount of sulphur in the fuel oil ,by controlling air heat temperature
by steam.Which increases the boiler efficiency as a consequence of
reducing the fuel consumption by 9% as a maximum , reduces heat
loss, minimizes maintenance costs and reduces the time of emergency
shutdowns.
The corrosion inhibition of transmision steel in 1M HCl solution in
the presence of Vanillin at temperature (20,30,40,50,60)°C at
concentration between (100-500) ppm for two hours were studied
using weight loss method.
