-
Seasonal Entropy, Diversity and Inequality Measures of Submitted and Accepted Papers Distributions In Peer-Reviewed Journals
Authors:
Marcel Ausloos,
Olgica Nedic,
Aleksandar Dekanski
Abstract:
This paper presents a novel method for finding features in the analysis of variable distributions stemming from time series. We apply the methodology to the case of submitted and accepted papers in peer-reviewed journals. We provide a comparative study of editorial decisions for papers submitted to two peer-reviewed journals: the Journal of the Serbian Chemical Society (JSCS) and this MDPI Entropy…
▽ More
This paper presents a novel method for finding features in the analysis of variable distributions stemming from time series. We apply the methodology to the case of submitted and accepted papers in peer-reviewed journals. We provide a comparative study of editorial decisions for papers submitted to two peer-reviewed journals: the Journal of the Serbian Chemical Society (JSCS) and this MDPI Entropy journal. We cover three recent years for which the fate of submitted papers, about 600 papers to JSCS and 2500 to Entropy, is completely determined. Instead of comparing the number distributions of these papers as a function of time with respect to a uniform distribution, we analyze the relevant probabilities, from which we derive the information entropy. It is argued that such probabilities are indeed more relevant for authors than the actual number of submissions. We tie this entropy analysis to the so called diversity of the variable distributions. Furthermore, we emphasize the correspondence between the entropy and the diversity with inequality measures, like the Herfindahl-Hirschman index and the Theil index, itself being in the class of entropy measures; the Gini coefficient which also measures the diversity in ranking is calculated for further discussion. In this sample, the seasonal aspects of the peer review process are outlined. It is found that the use of such indices, non linear transformations of the data distributions, allow to distinguish features and evolutions of peer review process as a function of time as well as comparing non-uniformity of distributions. Furthermore, t- and z- statistical tests are applied in order to measure the significance (p-level) of the findings, i.e. whether papers are more likely to be accepted if they are submitted during a few specific months or "season"; the predictability strength depends on the journal.
△ Less
Submitted 13 October, 2019;
originally announced October 2019.
-
Efficiency in managing peer-review of scientific manuscripts -- editors' perspective
Authors:
Olgica Nedic,
Ivana Drvenica,
Marcel Ausloos,
Aleksandar Dekanski
Abstract:
The purpose of this paper is to introduce a model for measuring the efficiency in managing peer-review of scientific manuscripts by editors. The approach employed is based on the assumption that the editorial aim is to manage publication with high efficiency, employing the least amount of editorial resources. Efficiency is defined in this research as a measure based on 7 variables. An on-line surv…
▽ More
The purpose of this paper is to introduce a model for measuring the efficiency in managing peer-review of scientific manuscripts by editors. The approach employed is based on the assumption that the editorial aim is to manage publication with high efficiency, employing the least amount of editorial resources. Efficiency is defined in this research as a measure based on 7 variables. An on-line survey was constructed and editors of journals originating from Serbia regularly publishing articles in the field of chemistry were invited to participate. An evaluation of the model is given based on responses from 24 journals and 50 editors. With this investigation we aimed to contribute to our understanding of the peer-review process and, possibly, offer a tool to improve the "efficiency" in journal editing. The proposed protocol may be adapted by other journals in order to assess the managing potential of editors.
△ Less
Submitted 12 October, 2019;
originally announced October 2019.
-
Artificial intelligence in peer review: How can evolutionary computation support journal editors?
Authors:
Maciej J. Mrowinski,
Piotr Fronczak,
Agata Fronczak,
Marcel Ausloos,
Olgica Nedic
Abstract:
With the volume of manuscripts submitted for publication growing every year, the deficiencies of peer review (e.g. long review times) are becoming more apparent. Editorial strategies, sets of guidelines designed to speed up the process and reduce editors workloads, are treated as trade secrets by publishing houses and are not shared publicly. To improve the effectiveness of their strategies, edito…
▽ More
With the volume of manuscripts submitted for publication growing every year, the deficiencies of peer review (e.g. long review times) are becoming more apparent. Editorial strategies, sets of guidelines designed to speed up the process and reduce editors workloads, are treated as trade secrets by publishing houses and are not shared publicly. To improve the effectiveness of their strategies, editors in small publishing groups are faced with undertaking an iterative trial-and-error approach. We show that Cartesian Genetic Programming, a nature-inspired evolutionary algorithm, can dramatically improve editorial strategies. The artificially evolved strategy reduced the duration of the peer review process by 30%, without increasing the pool of reviewers (in comparison to a typical human-developed strategy). Evolutionary computation has typically been used in technological processes or biological ecosystems. Our results demonstrate that genetic programs can improve real-world social systems that are usually much harder to understand and control than physical systems.
△ Less
Submitted 2 December, 2017;
originally announced December 2017.
-
Day of the week effect in paper submission/acceptance/rejection to/in/by peer review journals. II. An ARCH econometric-like modeling
Authors:
Marcel Ausloos,
Olgica Nedic,
Aleksandar Dekanski,
Maciej J. Mrowinski,
Piotr Fronczak,
Agata Fronczak
Abstract:
This paper aims at providing a statistical model for the preferred behavior of authors submitting a paper to a scientific journal. The electronic submission of (about 600) papers to the Journal of the Serbian Chemical Society has been recorded for every day from Jan. 01, 2013 till Dec. 31, 2014, together with the acceptance or rejection paper fate. Seasonal effects and editor roles (through desk r…
▽ More
This paper aims at providing a statistical model for the preferred behavior of authors submitting a paper to a scientific journal. The electronic submission of (about 600) papers to the Journal of the Serbian Chemical Society has been recorded for every day from Jan. 01, 2013 till Dec. 31, 2014, together with the acceptance or rejection paper fate. Seasonal effects and editor roles (through desk rejection and subfield editors) are examined. An ARCH-like econometric model is derived stressing the main determinants of the favorite day-of-week process.
△ Less
Submitted 14 November, 2016;
originally announced November 2016.
-
Day of the week effect in paper submission/acceptance/rejection to/in/by peer review journals
Authors:
Marcel Ausloos,
Olgica Nedic,
Aleksandar Dekanski
Abstract:
This paper aims at providing an introduction to the behavior of authors submitting a paper to a scientific journal. Dates of electronic submission of papers to the Journal of the Serbian Chemical Society have been recorded from the 1st January 2013 till the 31st December 2014, thus over 2 years.
There is no Monday or Friday effect like in financial markets, but rather a Tuesday-Wednesday effect…
▽ More
This paper aims at providing an introduction to the behavior of authors submitting a paper to a scientific journal. Dates of electronic submission of papers to the Journal of the Serbian Chemical Society have been recorded from the 1st January 2013 till the 31st December 2014, thus over 2 years.
There is no Monday or Friday effect like in financial markets, but rather a Tuesday-Wednesday effect occurs: papers are more often submitted on Wednesday; however, the relative number of going to be accepted papers is larger if these are submitted on Tuesday. On the other hand, weekend days (Saturday and Sunday) are not the best days to finalize and submit manuscripts. An interpretation based on the type of submitted work ("experimental chemistry") and on the influence of (senior) coauthors is presented. A thermodynamic connection is proposed within an entropy context. A (new) entropic distance is defined in order to measure the "opaqueness" = disorder) of the submission process.
△ Less
Submitted 6 April, 2016;
originally announced April 2016.
-
Quantifying the quality of peer reviewers through Zipf's law
Authors:
Marcel Ausloos,
Olgica Nedic,
Agata Fronczak,
Piotr Fronczak
Abstract:
This paper introduces a statistical and other analysis of peer reviewers in order to approach their "quality" through some quantification measure, thereby leading to some quality metrics. Peer reviewer reports for the Journal of the Serbian Chemical Society are examined. The text of each report has first to be adapted to word counting software in order to avoid jargon inducing confusion when searc…
▽ More
This paper introduces a statistical and other analysis of peer reviewers in order to approach their "quality" through some quantification measure, thereby leading to some quality metrics. Peer reviewer reports for the Journal of the Serbian Chemical Society are examined. The text of each report has first to be adapted to word counting software in order to avoid jargon inducing confusion when searching for the word frequency: e.g. C must be distinguished, depending if it means Carbon or Celsius, etc. Thus, every report has to be carefully "rewritten". Thereafter, the quantity, variety and distribution of words are examined in each report and compared to the whole set. Two separate months, according when reports came in, are distinguished to observe any possible hidden spurious effects. Coherence is found. An empirical distribution is searched for through a Zipf-Pareto rank-size law. It is observed that peer review reports are very far from usual texts in this respect. Deviations from the usual (first) Zipf's law are discussed. A theoretical suggestion for the "best (or worst) report" and by extension "good (or bad) reviewer", within this context, is provided from an entropy argument, through the concept of "distance to average" behavior. Another entropy-based measure also allows to measure the journal reviews (whence reviewers) for further comparison with other journals through their own reviewer reports.
△ Less
Submitted 23 August, 2015;
originally announced August 2015.
-
Review times in peer review: quantitative analysis of editorial workflows
Authors:
Maciej J. Mrowinski,
Agata Fronczak,
Piotr Fronczak,
Olgica Nedic,
Marcel Ausloos
Abstract:
We examine selected aspects of peer review and suggest possible improvements. To this end, we analyse a dataset containing information about 300 papers submitted to the Biochemistry and Biotechnology section of the Journal of the Serbian Chemical Society. After separating the peer review process into stages that each review has to go through, we use a weighted directed graph to describe it in a pr…
▽ More
We examine selected aspects of peer review and suggest possible improvements. To this end, we analyse a dataset containing information about 300 papers submitted to the Biochemistry and Biotechnology section of the Journal of the Serbian Chemical Society. After separating the peer review process into stages that each review has to go through, we use a weighted directed graph to describe it in a probabilistic manner and test the impact of some modifications of the editorial policy on the efficiency of the whole process.
△ Less
Submitted 5 August, 2015;
originally announced August 2015.
