Large, pre-trained transformer language models, which are pervasive in natural language processing tasks, are notoriously expensive to train. To reduce the cost of training such large models, prior work has developed smaller, more compact models whic
h achieves a significant speedup in training time while maintaining competitive accuracy to the original model on downstream tasks. Though these smaller pre-trained models have been widely adopted by the community, it is not known how well are they calibrated compared to their larger counterparts. In this paper, focusing on a wide range of tasks, we thoroughly investigate the calibration properties of pre-trained transformers, as a function of their size. We demonstrate that when evaluated in-domain, smaller models are able to achieve competitive, and often better, calibration compared to larger models, while achieving significant speedup in training time. Post-hoc calibration techniques further reduce calibration error for all models in-domain. However, when evaluated out-of-domain, larger models tend to be better calibrated, and label-smoothing instead is an effective strategy to calibrate models in this setting.
Natural language inference requires reasoning about contradictions, negations, and their commonsense implications. Given a simple premise (e.g., I'm mad at you''), humans can reason about the varying shades of contradictory statements ranging from st
raightforward negations (I'm not mad at you'') to commonsense contradictions (I'm happy''). Moreover, these negated or contradictory statements shift the commonsense implications of the original premise in interesting and nontrivial ways. For example, while I'm mad'' implies I'm unhappy about something,'' negating the premise does not necessarily negate the corresponding commonsense implications. In this paper, we present the first comprehensive study focusing on commonsense implications of negated statements and contradictions. We introduce ANION, a new commonsense knowledge graph with 624K if-then rules focusing on negated and contradictory events. We then present joint generative and discriminative inference models for this new resource, providing novel empirical insights on how logical negations and commonsense contradictions reshape the commonsense implications of their original premises.
We consider the problem of using observational data to estimate the causal effects of linguistic properties. For example, does writing a complaint politely lead to a faster response time? How much will a positive product review increase sales? This p
aper addresses two technical challenges related to the problem before developing a practical method. First, we formalize the causal quantity of interest as the effect of a writer's intent, and establish the assumptions necessary to identify this from observational data. Second, in practice, we only have access to noisy proxies for the linguistic properties of interest---e.g., predictions from classifiers and lexicons. We propose an estimator for this setting and prove that its bias is bounded when we perform an adjustment for the text. Based on these results, we introduce TextCause, an algorithm for estimating causal effects of linguistic properties. The method leverages (1) distant supervision to improve the quality of noisy proxies, and (2) a pre-trained language model (BERT) to adjust for the text. We show that the proposed method outperforms related approaches when estimating the effect of Amazon review sentiment on semi-simulated sales figures. Finally, we present an applied case study investigating the effects of complaint politeness on bureaucratic response times.
In deployment, systems that use speech as input must make use of automated transcriptions. Yet, typically when these systems are evaluated, gold transcriptions are assumed. We explicitly examine the impact of transcription errors on the downstream pe
rformance of a multi-modal system on three related tasks from three datasets: emotion, sarcasm, and personality detection. We include three separate transcription tools and show that while all automated transcriptions propagate errors that substantially impact downstream performance, the open-source tools fair worse than the paid tool, though not always straightforwardly, and word error rates do not correlate well with downstream performance. We further find that the inclusion of audio features partially mitigates transcription errors, but that a naive usage of a multi-task setup does not.
This study investigates the Monthly Effect (Semi-Month Effect
and Turn of the Month Effect) in the stock markets of Egypt,
Jordan, Iraq and Syria, over the period from January 2010 to
December 2014. The daily returns for each index were analyzed
by using T test or Mann-Whitney test. Moreover, the regression
analysis was also applied.
The Contract of Paper Collection forces many of Contractual obligations on parties, and bank's customer may suffer damages because of the bank's contractual liability in this issue. Hence, it is important to determine the parties' obligations and the
scope of bank liability, since the Syrian commercial law did not regulate the codes of this contract nor the scope of banks liability .So it's necessary to discuss the provisions of the agency contract and general principles of liability in the Syrian civil law, taking into consideration the rules of banking customs
The codes for sustainable homes have been introduced to drive a step-change in green building practice. It
set the international standards for key elements of the design, construction and planning. It is a tool for
designers and structural engineer
s and real estate developers and users. This paper reviews the concept of
sustainability codes through some of the world's leading experiments and implementation mechanisms . In
order to reach the main points for each codes and to find out what is essential and what is linked to the
characteristics of the region, this paper also present a brief explanation for the codes and limitations
imposed on the building through the evaluation process in the international codes. The paper also
explained degrees of classification and its effectiveness on a building, whether its binding or optional.
This book talks about the history of Lattakia Governorate, and its exposition of the archaeological sites in it. The history of the governorate was written for the first time by the French occupation, which contains many shortcomings that make it inc
omplete. The history written in this book includes the most important archaeological discoveries and brings together many Western studies.
With regard to antiquities, we can say that it constitutes the first comprehensive study of the various archaeological sites in Lattakia Governorate.
