Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userOn the Predictability of Future Impact in Science
147
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Correctly assessing a scientists past research impact and potential for future impact is key in recruitment decisions and other evaluation processes. While a candidates future impact is the main concern for these decisions, most measures only quantify the impact of previous work. Recently, it has been argued that linear regression models are capable of predicting a scientists future impact. By applying that future impact model to 762 careers drawn from three disciplines: physics, biology, and mathematics, we identify a number of subtle, but critical, flaws in current models. Specifically, cumulative non-decreasing measures like the h-index contain intrinsic autocorrelation, resulting in significant overestimation of their predictive power. Moreover, the predictive power of these models depend heavily upon scientists career age, producing least accurate estimates for young researchers. Our results place in doubt the suitability of such models, and indicate further investigation is required before they can be used in recruiting decisions.
rate research
Read More
John Desmond Bernal (1901-1970) was one of the most eminent scientists in molecular biology, and also regarded as the founding father of the Science of Science. His book The Social Function of Science laid the theoretical foundations for the discipline. In this article, we summarize four chief characteristics of his ideas in the Science of Science: the socio-historical perspective, theoretical models, qualitative and quantitative approaches, and studies of science planning and policy. China has constantly reformed its scientific and technological system based on research evidence of the Science of Science. Therefore, we analyze the impact of Bernal Science-of-Science thoughts on the development of Science of Science in China, and discuss how they might be usefully taken still further in quantitative studies of science.
We stress-test the career predictability model proposed by Acuna et al. [Nature 489, 201-202 2012] by applying their model to a longitudinal career data set of 100 Assistant professors in physics, two from each of the top 50 physics departments in the US. The Acuna model claims to predict h(t+Delta t), a scientists h-index Delta t years into the future, using a linear combination of 5 cumulative career measures taken at career age t. Here we investigate how the predictability depends on the aggregation of career data across multiple age cohorts. We confirm that the Acuna model does a respectable job of predicting h(t+Delta t) up to roughly 6 years into the future when aggregating all age cohorts together. However, when calculated using subsets of specific age cohorts (e.g. using data for only t=3), we find that the models predictive power significantly decreases, especially when applied to early career years. For young careers, the model does a much worse job of predicting future impact, and hence, exposes a serious limitation. The limitation is particularly concerning as early career decisions make up a significant portion, if not the majority, of cases where quantitative approaches are likely to be applied.
Analyzing a large data set of publications drawn from the most competitive journals in the natural and social sciences we show that research careers exhibit the broad distributions of individual achievement characteristic of systems in which cumulative advantage plays a key role. While most researchers are personally aware of the competition implicit in the publication process, little is known about the levels of inequality at the level of individual researchers. We analyzed both productivity and impact measures for a large set of researchers publishing in high-impact journals. For each researcher cohort we calculated Gini inequality coefficients, with average Gini values around 0.48 for total publications and 0.73 for total citations. For perspective, these observed values are well in excess of the inequality levels observed for personal income in developing countries. Investigating possible sources of this inequality, we identify two potential mechanisms that act at the level of the individual that may play defining roles in the emergence of the broad productivity and impact distributions found in science. First, we show that the average time interval between a researchers successive publications in top journals decreases with each subsequent publication. Second, after controlling for the time dependent features of citation distributions, we compare the citation impact of subsequent publications within a researchers publication record. We find that as researchers continue to publish in top journals, there is more likely to be a decreasing trend in the relative citation impact with each subsequent publication. This pattern highlights the difficulty of repeatedly publishing high-impact research and the intriguing possibility that confirmation bias plays a role in the evaluation of scientific careers.
The birth and decline of disciplines are critical to science and society. However, no quantitative model to date allows us to validate competing theories of whether the emergence of scientific disciplines drives or follows the formation of social communities of scholars. Here we propose an agent-based model based on a emph{social dynamics of science,} in which the evolution of disciplines is guided mainly by the social interactions among scientists. We find that such a social theory can account for a number of stylized facts about the relationships between disciplines, authors, and publications. These results provide strong quantitative support for the key role of social interactions in shaping the dynamics of science. A science of science must gauge the role of exogenous events, such as scientific discoveries and technological advances, against this purely social baseline.
We perform the analysis of scientific collaboration at the level of universities. The scope of this study is to answer two fundamental questions: (i) can one indicate a category (i.e., a scientific discipline) that has the greatest impact on the rank of the university and (ii) do the best universities collaborate with the best ones only? Using two university ranking lists (ARWU and QS) as well as data from the Science Citation Index we show how the number of publications in certain categories correlates with the university rank. Moreover, using complex networks analysis, we give hints that the scientific collaboration is highly embedded in the physical space and the number of common papers decays with the distance between them. We also show the strength of the ties between universities is proportional to product of their total number of publications.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
