-
A two-stage model for factors influencing citation counts
Authors:
Pablo Dorta-González,
Emilio Gómez-Déniz
Abstract:
This work aims to study a count response random variable, the number of citations of a research paper, affected by some explanatory variables through a suitable regression model. Due to the fact that the count variable exhibits substantial variation since the sample variance is larger than the sample mean, the classical Poisson regression model seems not to be appropriate. We concentrate attention…
▽ More
This work aims to study a count response random variable, the number of citations of a research paper, affected by some explanatory variables through a suitable regression model. Due to the fact that the count variable exhibits substantial variation since the sample variance is larger than the sample mean, the classical Poisson regression model seems not to be appropriate. We concentrate attention on the negative binomial regression model, which allows the variance of each measurement to be a function of its predicted value. Nevertheless, the process of citations of papers may be divided into two parts. In the first stage, the paper has no citations, and the second part provides the intensity of the citations. A hurdle model for separating the documents with citations and those without citations is considered. The dataset for the empirical application consisted of 43,190 research papers in the field of Economics and Business from 2014-2021, obtained from The Lens database. Citation counts and social attention scores for each article were gathered from Altmetric database. The main findings indicate that both collaboration and funding have a positive impact on citation counts and reduce the likelihood of receiving zero citations. Higher journal impact factors lead to higher citation counts, while lower peer review ratings lead to fewer citations and a higher probability of zero citations. Mentions in news, blogs, and social media have varying but generally limited effects on citation counts. Open access via repositories (green OA) correlates with higher citation counts and a lower probability of zero citations. In contrast, OA via the publisher's website without an explicit open license (bronze OA) is associated with higher citation counts but also with a higher probability of zero citations.
△ Less
Submitted 21 May, 2025;
originally announced May 2025.
-
Effect of perceived preprint effectiveness and research intensity on posting behaviour
Authors:
Pablo Dorta-González,
María Isabel Dorta-González
Abstract:
Open science is increasingly recognised worldwide, with preprint posting emerging as a key strategy. This study explores the factors influencing researchers' adoption of preprint publication, particularly the perceived effectiveness of this practice and research intensity indicators such as publication and review frequency. Using open data from a comprehensive survey with 5,873 valid responses, we…
▽ More
Open science is increasingly recognised worldwide, with preprint posting emerging as a key strategy. This study explores the factors influencing researchers' adoption of preprint publication, particularly the perceived effectiveness of this practice and research intensity indicators such as publication and review frequency. Using open data from a comprehensive survey with 5,873 valid responses, we conducted regression analyses to control for demographic variables. Researchers' productivity, particularly the number of journal articles and books published, greatly influences the frequency of preprint deposits. The perception of the effectiveness of preprints follows this. Preprints are viewed positively in terms of early access to new research, but negatively in terms of early feedback. Demographic variables, such as gender and the type of organisation conducting the research, do not have a significant impact on the production of preprints when other factors are controlled for. However, the researcher's discipline, years of experience and geographical region generally have a moderate effect on the production of preprints. These findings highlight the motivations and barriers associated with preprint publication and provide insights into how researchers perceive the benefits and challenges of this practice within the broader context of open science.
△ Less
Submitted 26 April, 2025;
originally announced April 2025.
-
Which kind of research papers influence policymaking
Authors:
Pablo Dorta-González
Abstract:
This study examines the use of evidence in policymaking by analysing a range of journal and article attributes, as well as online engagement metrics. It employs a large-scale citation analysis of nearly 150,000 articles covering diverse policy topics. The findings highlight that scholarly citations exert the strongest positive influence on policy citations. Articles from journals with a higher cit…
▽ More
This study examines the use of evidence in policymaking by analysing a range of journal and article attributes, as well as online engagement metrics. It employs a large-scale citation analysis of nearly 150,000 articles covering diverse policy topics. The findings highlight that scholarly citations exert the strongest positive influence on policy citations. Articles from journals with a higher citation impact and larger Mendeley readership are cited more frequently in policy documents. Other online engagements, such as news and blog mentions, also boost policy citations, while mentions on social media X have a negative effect. The finding that highly cited and widely read papers are also frequently referenced in policy documents likely reflects the perception among policymakers that such research is more trustworthy. In contrast, papers that derive their influence primarily from social media tend to be cited less often in policy contexts.
△ Less
Submitted 26 April, 2025;
originally announced April 2025.
-
Linking Science and Industry: Influence of Scientific Research on Technological Innovation through Patent Citations
Authors:
Pablo Dorta-González,
Alejandro Rodríguez-Caro,
María Isabel Dorta-González
Abstract:
This study explores the connection between patent citations and scientific publications across six fields: Biochemistry, Genetics, Pharmacology, Engineering, Mathematics, and Physics. Analysing 117,590 papers from 2014 to 2023, the research emphasises how publication year, open access (OA) status, and discipline influence patent citations. Openly accessible papers, particularly those in hybrid OA…
▽ More
This study explores the connection between patent citations and scientific publications across six fields: Biochemistry, Genetics, Pharmacology, Engineering, Mathematics, and Physics. Analysing 117,590 papers from 2014 to 2023, the research emphasises how publication year, open access (OA) status, and discipline influence patent citations. Openly accessible papers, particularly those in hybrid OA journals or green OA repositories, are significantly more likely to be cited in patents, seven times more than those mentioned in blogs, and over twice as likely compared to older publications. However, papers with policy-related references are less frequently cited, indicating that patents may prioritise commercially viable innovations over those addressing societal challenges. Disciplinary differences reveal distinct innovation patterns across sectors. While academic visibility via blogs or platforms like Mendeley increases within scholarly circles, these have limited impact on patent citations. The study also finds that increased funding, possibly tied to applied research trends and fully open access journals, negatively affects patent citations. Social media presence and the number of authors have minimal impact. These findings highlight the complex factors shaping the integration of scientific research into technological innovations.
△ Less
Submitted 20 February, 2025;
originally announced February 2025.
-
Generative artificial intelligence usage by researchers at work: Effects of gender, career stage, type of workplace, and perceived barriers
Authors:
Pablo Dorta-González,
Alexis Jorge López-Puig,
María Isabel Dorta-González,
Sara M. González-Betancor
Abstract:
The integration of generative artificial intelligence technology into research environments has become increasingly common in recent years, representing a significant shift in the way researchers approach their work. This paper seeks to explore the factors underlying the frequency of use of generative AI amongst researchers in their professional environments. As survey data may be influenced by a…
▽ More
The integration of generative artificial intelligence technology into research environments has become increasingly common in recent years, representing a significant shift in the way researchers approach their work. This paper seeks to explore the factors underlying the frequency of use of generative AI amongst researchers in their professional environments. As survey data may be influenced by a bias towards scientists interested in AI, potentially skewing the results towards the perspectives of these researchers, this study uses a regression model to isolate the impact of specific factors such as gender, career stage, type of workplace, and perceived barriers to using AI technology on the frequency of use of generative AI. It also controls for other relevant variables such as direct involvement in AI research or development, collaboration with AI companies, geographic location, and scientific discipline. Our results show that researchers who face barriers to AI adoption experience an 11% increase in tool use, while those who cite insufficient training resources experience an 8% decrease. Female researchers experience a 7% decrease in AI tool usage compared to men, while advanced career researchers experience a significant 19% decrease. Researchers associated with government advisory groups are 45% more likely to use AI tools frequently than those in government roles. Researchers in for-profit companies show an increase of 19%, while those in medical research institutions and hospitals show an increase of 16% and 15%, respectively. This paper contributes to a deeper understanding of the mechanisms driving the use of generative AI tools amongst researchers, with valuable implications for both academia and industry.
△ Less
Submitted 31 August, 2024;
originally announced September 2024.
-
Societal and scientific impact of policy research: A large-scale empirical study of some explanatory factors using Altmetric and Overton
Authors:
Pablo Dorta-González,
Alejandro Rodríguez-Caro,
María Isabel Dorta-González
Abstract:
This study investigates how scientific research influences policymaking by analyzing citations of research articles in policy documents (policy impact) for nearly 125,000 articles across 434 public policy journals. We reveal distinct citation patterns between policymakers and other stakeholders like researchers, journalists, and the public. News and blog mentions, social media engagement, and open…
▽ More
This study investigates how scientific research influences policymaking by analyzing citations of research articles in policy documents (policy impact) for nearly 125,000 articles across 434 public policy journals. We reveal distinct citation patterns between policymakers and other stakeholders like researchers, journalists, and the public. News and blog mentions, social media engagement, and open access publications (excluding fully open access) significantly increase the likelihood of a research article being cited in policy documents. Conversely, articles locked behind paywalls and those published under the full open access model (based on Altmetric data) have a lower chance of being policy-cited. Publication year and policy type show no significant influence. Our findings emphasize the crucial role of science communication channels like news media and social media in bridging the gap between research and policy. Interestingly, academic citations hold a weaker influence on policy citations compared to news mentions, suggesting a potential disconnect between how researchers reference research and how policymakers utilize it. This highlights the need for improved communication strategies to ensure research informs policy decisions more effectively. This study provides valuable insights for researchers, policymakers, and science communicators. Researchers can tailor their dissemination efforts to reach policymakers through media channels. Policymakers can leverage these findings to identify research with higher policy relevance. Science communicators can play a critical role in translating research for policymakers and fostering dialogue between the scientific and policymaking communities.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
A field- and time-normalized Bayesian approach to measuring the impact of a publication
Authors:
Emilio Gómez-Déniz,
Pablo Dorta-González
Abstract:
Measuring the impact of a publication in a fair way is a significant challenge in bibliometrics, as it must not introduce biases between fields and should enable comparison of the impact of publications from different years. In this paper, we propose a Bayesian approach to tackle this problem, motivated by empirical data demonstrating heterogeneity in citation distributions. The approach uses the…
▽ More
Measuring the impact of a publication in a fair way is a significant challenge in bibliometrics, as it must not introduce biases between fields and should enable comparison of the impact of publications from different years. In this paper, we propose a Bayesian approach to tackle this problem, motivated by empirical data demonstrating heterogeneity in citation distributions. The approach uses the a priori distribution of citations in each field to estimate the expected a posteriori distribution in that field. This distribution is then employed to normalize the citations received by a publication in that field. Our main contribution is the Bayesian Impact Score, a measure of the impact of a publication. This score is increasing and concave with the number of citations received and decreasing and convex with the age of the publication. This means that the marginal score of an additional citation decreases as the cumulative number of citations increases and increases as the time since publication of the document grows. Finally, we present an empirical application of our approach in eight subject categories using the Scopus database and a comparison with the normalized impact indicator Field Citation Ratio from the Dimensions AI database.
△ Less
Submitted 6 March, 2024;
originally announced March 2024.
-
Modeling citation concentration through a mixture of Leimkuhler curves
Authors:
Emilio Gómez-Déniz,
Pablo Dorta-González
Abstract:
When a graphical representation of the cumulative percentage of total citations to articles, ordered from most cited to least cited, is plotted against the cumulative percentage of articles, we obtain a Leimkuhler curve. In this study, we noticed that standard Leimkuhler functions may not be sufficient to provide accurate fits to various empirical informetrics data. Therefore, we introduce a new a…
▽ More
When a graphical representation of the cumulative percentage of total citations to articles, ordered from most cited to least cited, is plotted against the cumulative percentage of articles, we obtain a Leimkuhler curve. In this study, we noticed that standard Leimkuhler functions may not be sufficient to provide accurate fits to various empirical informetrics data. Therefore, we introduce a new approach to Leimkuhler curves by fitting a known probability density function to the initial Leimkuhler curve, taking into account the presence of a heterogeneity factor. As a significant contribution to the existing literature, we introduce a pair of mixture distributions (called PG and PIG) to bibliometrics. In addition, we present closed-form expressions for Leimkuhler curves. {Some measures of citation concentration are examined empirically for the basic models (based on the Power {and Pareto distributions}) and the mixed models derived from {these}.} An application to two sources of informetric data was conducted to see how the mixing models outperform the standard basic models. The different models were fitted using non-linear least squares estimation.
△ Less
Submitted 13 January, 2024;
originally announced January 2024.
-
Does society show differential attention to researchers based on gender and field?
Authors:
Sara M. González-Betancor,
Pablo Dorta-González
Abstract:
While not all researchers prioritize social impact, it is undeniably a crucial aspect that adds significance to their work. The objective of this paper is to explore potential gender differences in the social attention paid to researchers and to examine their association with specific fields of study. To achieve this goal, the paper analyzes four dimensions of social influence and examines three m…
▽ More
While not all researchers prioritize social impact, it is undeniably a crucial aspect that adds significance to their work. The objective of this paper is to explore potential gender differences in the social attention paid to researchers and to examine their association with specific fields of study. To achieve this goal, the paper analyzes four dimensions of social influence and examines three measures of social attention to researchers. The dimensions are media influence (mentions in mainstream news), political influence (mentions in public policy reports), social media influence (mentions in Twitter), and educational influence (mentions in Wikipedia). The measures of social attention to researchers are: proportion of publications with social mentions (social attention orientation), mentions per publication (level of social attention), and mentions per mentioned publication (intensity of social attention). By analyzing the rankings of authors -- for the four dimensions with the three measures in the 22 research fields of the Web of Science database -- and by using Spearman correlation coefficients, we conclude that: 1) significant differences are observed between fields; 2) the dimensions capture different and independent aspects of the social impact. Finally, we use non-parametric means comparison tests to detect gender bias in social attention. We conclude that for most fields and dimensions with enough non-zero altmetrics data, gender differences in social attention are not predominant, but are still present and vary across fields.
△ Less
Submitted 22 August, 2023;
originally announced August 2023.
-
The funding effect on citation and social attention: the UN Sustainable Development Goals (SDGs) as a case study
Authors:
Pablo Dorta-González,
María Isabel Dorta-González
Abstract:
Purpose: Academic citation and social attention measure different dimensions in the impact of research results. We quantify the contribution of funding to both indicators considering the differences attributable to the research field and access type. Design/methodology/approach: Citation and social attention accumulated until the year 2021 of more than 367 thousand research articles published in t…
▽ More
Purpose: Academic citation and social attention measure different dimensions in the impact of research results. We quantify the contribution of funding to both indicators considering the differences attributable to the research field and access type. Design/methodology/approach: Citation and social attention accumulated until the year 2021 of more than 367 thousand research articles published in the year 2018, are studied. We consider funding acknowledgements in the research articles. The data source is Dimensions and the units of study are research articles in the UN Sustainable Development Goals. Findings: Most cited goals by researchers do not coincide with those that arouse greater social attention. A small proportion of articles accumulates a large part of the citations and most of the social attention. Both citation and social attention grow with funding. Thus, funded research has a greater probability of being cited in academic articles and mentioned in social media. Funded research receives on average two to three times more citations and 2.5 to 4.5 times more social attention than unfunded research. Moreover, the open access modalities gold and hybrid have the greatest advantages in citation and social attention due to funding. Originality: The joint evaluation of the effect of both funding and open access on social attention. Research limitations: Specific topics were studied in a specific period. Studying other topics and/or different time periods might result in different findings. Practical implications: When funding to publish in open or hybrid access journals is not available, it is advisable to self-archiving the pre-print or post-print version in a freely accessible repository. Social implications: Although cautiously, it is also advisable to consider the social impact of the research to complement the scientific impact in the evaluation of the research.
△ Less
Submitted 3 April, 2023;
originally announced April 2023.
-
A Multiple Linear Regression Analysis to Measure the Journal Contribution to the Social Attention of Research
Authors:
Pablo Dorta-González
Abstract:
This paper proposes a three-year average of social attention as a more reliable measure of social impact for journals, since the social attention of research can vary widely among scientific articles, even within the same journal. The proposed measure is used to evaluate a journal's contribution to social attention in comparison to other bibliometric indicators. The study uses Dimensions as a data…
▽ More
This paper proposes a three-year average of social attention as a more reliable measure of social impact for journals, since the social attention of research can vary widely among scientific articles, even within the same journal. The proposed measure is used to evaluate a journal's contribution to social attention in comparison to other bibliometric indicators. The study uses Dimensions as a data source and examines research articles from 76 disciplinary library and information science journals through multiple linear regression analysis. The study identifies socially influential journals whose contribution to social attention is twice that of scholarly impact as measured by citations. In addition, the study finds that the number of authors and open access have a moderate impact on social attention, while the journal impact factor has a negative impact and funding has a small impact.
△ Less
Submitted 23 March, 2023;
originally announced March 2023.
-
Influencing factors of Twitter mentions of scientific papers
Authors:
Pablo Dorta-González
Abstract:
Purpose: This paper explores some influencing factors of Twitter mentions of scientific research. The results can help to understand the relationships between various altmetrics. Design/methodology/approach: Data on research mentions in Altmetric and a multiple linear regression analysis are used. Findings: Among the variables analyzed, the number of mainstream news is the factor that most influen…
▽ More
Purpose: This paper explores some influencing factors of Twitter mentions of scientific research. The results can help to understand the relationships between various altmetrics. Design/methodology/approach: Data on research mentions in Altmetric and a multiple linear regression analysis are used. Findings: Among the variables analyzed, the number of mainstream news is the factor that most influences the number of mentions on Twitter, followed by the fact of dealing with a highly topical issue such as COVID-19. The influence is weaker in the case of expert recommendations and the consolidation of knowledge in the form of a review. The lowest influence corresponds to both the public policies through references in reports, and to citations in Wikipedia, while mentions in patent applications does not have a significant influence. Research limitations: A specific field was studied in a specific time frame. Studying other fields and/or different time periods might result in different findings. Practical implications: Governments increasingly push researchers toward activities with societal impact and this study can help understand how different factors affect social media attention. Originality/value: Understanding social media attention of research is essential when implementing societal impact indicators.
△ Less
Submitted 11 December, 2022; v1 submitted 21 October, 2022;
originally announced October 2022.
-
Collaboration Effect by Co-Authorship on Academic Citation and Social Attention of Research
Authors:
Pablo Dorta-González,
María Isabel Dorta-González
Abstract:
Academic citation and social attention measure different dimensions of the impact of research results. Both measures do not correlate with each other, and they are influenced by many factors. Among these factors are the field of research, the type of access, and co-authorship. In this study, the increase in the impact due to co-authorship in scientific articles disaggregated by field of research a…
▽ More
Academic citation and social attention measure different dimensions of the impact of research results. Both measures do not correlate with each other, and they are influenced by many factors. Among these factors are the field of research, the type of access, and co-authorship. In this study, the increase in the impact due to co-authorship in scientific articles disaggregated by field of research and access type, was quantified. For this, the citations and social attention accumulated until the year 2021 by a total of 244,880 research articles published in the year 2018, were analyzed. The data source was Dimensions.ai, and the units of study were research articles in Economics, History and Archaeology, and Mathematics. As the main results, a small proportion of the articles received a large part of the citations and most of the social attention. Both citations and social attention in-creased, in general, with the number of co-authors. Thus, the greater the number of co-authors, the greater the probability of being cited in academic articles and mentioned on social media. The advantage in citation and social attention due to collaboration is independent of the access type for the publication. Furthermore, although collaboration with an additional co-author is in general positive in terms of citation and social attention, these positive effects reduce as the number of co-authors increases.
△ Less
Submitted 15 June, 2022;
originally announced June 2022.
-
Modeling the obsolescence of research literature in disciplinary journals through the age of their cited references
Authors:
Pablo Dorta-González,
Emilio Gómez-Déniz
Abstract:
There are different citation habits in the research fields that influence the obsolescence of the research literature. We analyze the distinctive obsolescence of research literature in disciplinary journals in eight scientific subfields based on cited references distribution, as a synchronous approach. We use both Negative Binomial (NB) and Poisson distributions to capture this obsolescence. The c…
▽ More
There are different citation habits in the research fields that influence the obsolescence of the research literature. We analyze the distinctive obsolescence of research literature in disciplinary journals in eight scientific subfields based on cited references distribution, as a synchronous approach. We use both Negative Binomial (NB) and Poisson distributions to capture this obsolescence. The corpus being examined is published in 2019 and covers 22,559 papers citing 872,442 references. Moreover, three measures to analyze the tail of the distribution are proposed: (i) cited reference survival rate, (ii) cited reference mortality rate, and (iii) cited reference percentile. These measures are interesting because the tail of the distribution collects the behavior of the citations at the time when the document starts to get obsolete in the sense that it is little cited (used). As main conclusion, the differences observed in obsolescence are so important even between disciplinary journals in the same subfield, that it would be necessary to use some measure for the tail of the citation distribution, such as those proposed in this paper, when analyzing in an appropriate way the long time impact of a journal.
△ Less
Submitted 16 March, 2022;
originally announced March 2022.
-
The influence of funding on the Open Access citation advantage
Authors:
Pablo Dorta-González,
María Isabel Dorta-González
Abstract:
Some of the citation advantage in open access is likely due to more access allows more people to read and hence cite articles they otherwise would not. However, causation is difficult to establish and there are many possible bias. Several factors can affect the observed differences in citation rates. Funder mandates can be one of them. Funders are likely to have OA requirement, and well-funded stu…
▽ More
Some of the citation advantage in open access is likely due to more access allows more people to read and hence cite articles they otherwise would not. However, causation is difficult to establish and there are many possible bias. Several factors can affect the observed differences in citation rates. Funder mandates can be one of them. Funders are likely to have OA requirement, and well-funded studies are more likely to receive more citations than poorly funded studies. In this paper this hypothesis is tested. Thus, we studied the effect of funding on the publication modality and the citations received in more than 128 thousand research articles, of which 31% were funded. These research articles come from 40 randomly selected subject categories in the year 2016, and the citations received from the period 2016-2020 in the Scopus database. We found open articles published in hybrid journals were considerably more cited than those in open access journals. Thus, articles under the hybrid gold modality are cite on average twice as those in the gold modality. This is the case regardless of funding, so this evidence is strong. Moreover, within the same publication modality, we found that funded articles generally obtain 50% more citations than unfunded ones. The most cited modality is the hybrid gold and the least cited is the gold, well below even the paywalled. Furthermore, the use of open access repositories considerably increases the citations received, especially for those articles without funding. Thus, the articles in open access repositories (green) are 50% more cited than the paywalled ones. This evidence is remarkable and does not depend on funding. Excluding the gold modality, there is a citation advantage in more than 75% of the cases and it is considerably greater among unfunded articles. This result is strong both across fields and over time.
△ Less
Submitted 4 February, 2022;
originally announced February 2022.
-
Contribution of the Open Access modality to the impact of hybrid journals controlling by field and time effects
Authors:
Pablo Dorta-González,
María Isabel Dorta-González
Abstract:
Researchers are more likely to read and cite papers to which they have access than those that they cannot obtain. Thus, the objective of this work is to analyze the contribution of the Open Access (OA) modality to the impact of hybrid journals. For this, the research articles in the year 2017 from 200 hybrid journals in four subject areas, and the citations received by such articles in the period…
▽ More
Researchers are more likely to read and cite papers to which they have access than those that they cannot obtain. Thus, the objective of this work is to analyze the contribution of the Open Access (OA) modality to the impact of hybrid journals. For this, the research articles in the year 2017 from 200 hybrid journals in four subject areas, and the citations received by such articles in the period 2017-2020 in the Scopus database, were analyzed. The journals were randomly selected from those with share of OA papers higher than some minimal value. More than 60 thousand research articles were analyzed in the sample, of which 24% under the OA modality. As results, we obtain that cites per article in both hybrid modalities strongly correlate. However, there is no correlation between the OA prevalence and cites per article in any of the hybrid modalities. There is OA citation advantage in 80% of hybrid journals. Moreover, the OA citation advantage is consistent across fields and held in time. We obtain an OA citation advantage of 50% in average, and higher than 37% in half of the hybrid journals. Finally, the OA citation advantage is higher in Humanities than in Science and Social Science.
△ Less
Submitted 23 January, 2022;
originally announced January 2022.
-
To what extent is researchers' data-sharing motivated by formal mechanisms of recognition and credit?
Authors:
Pablo Dorta-González,
Sara M. González-Betancor,
María Isabel Dorta-González
Abstract:
Data sharing by researchers is a centerpiece of Open Science principles and scientific progress. For a sample of 6019 researchers, we analyze the extent/frequency of their data sharing. Specifically, the relationship with the following four variables: how much they value data citations, the extent to which their data-sharing activities are formally recognized, their perceptions of whether sufficie…
▽ More
Data sharing by researchers is a centerpiece of Open Science principles and scientific progress. For a sample of 6019 researchers, we analyze the extent/frequency of their data sharing. Specifically, the relationship with the following four variables: how much they value data citations, the extent to which their data-sharing activities are formally recognized, their perceptions of whether sufficient credit is awarded for data sharing, and the reported extent to which data citations motivate their data sharing. In addition, we analyze the extent to which researchers have reused openly accessible data, as well as how data sharing varies by professional age-cohort, and its relationship to the value they place on data citations. Furthermore, we consider most of the explanatory variables simultaneously by estimating a multiple linear regression that predicts the extent/frequency of their data sharing. We use the dataset of the State of Open Data Survey 2019 by Springer Nature and Digital Science. Results do allow us to conclude that a desire for recognition/credit is a major incentive for data sharing. Thus, the possibility of receiving data citations is highly valued when sharing data, especially among younger researchers, irrespective of the frequency with which it is practiced. Finally, the practice of data sharing was found to be more prevalent at late research career stages, despite this being when citations are less valued and have a lower motivational impact. This could be due to the fact that later-career researchers may benefit less from keeping their data private.
△ Less
Submitted 14 January, 2021;
originally announced January 2021.
-
Risk of Interruption of Doctoral Studies and Mental Health in PhD Students
Authors:
Sara M. González-Betancor,
Pablo Dorta-González
Abstract:
PhD students report a higher prevalence of mental illness symptoms than highly educated individuals in the general population. This situation presents a serious problem for universities. Thus, the knowledge about this phenomenon is of great importance in decision-making. In this paper we use the Nature PhD survey 2019 and estimate several binomial logistic regression models to analyze the risk of…
▽ More
PhD students report a higher prevalence of mental illness symptoms than highly educated individuals in the general population. This situation presents a serious problem for universities. Thus, the knowledge about this phenomenon is of great importance in decision-making. In this paper we use the Nature PhD survey 2019 and estimate several binomial logistic regression models to analyze the risk of interrupting doctoral studies. This risk is measured through the desire of change in either the supervisor or the area of expertise, or the wish of not pursue a PhD. Among the explanatory factors, we focus on the influence of anxiety/depression, discrimination, and bullying. As control variables we use demographic characteristics and others related with the doctoral program. Insufficient contact time with supervisors, and exceeding time spent studying -crossing the 50-h week barrier-, are risk factors of PhD studies interruption, but the most decisive risk factor is poor mental health. Universities should therefore foster an environment of well-being, which allows the development of autonomy and resilience of their PhD students or, when necessary, which fosters the development of conflict resolution skills.
△ Less
Submitted 28 September, 2020;
originally announced October 2020.
-
Open Access effect on uncitedness: A large-scale study controlling by discipline, source type and visibility
Authors:
Pablo Dorta-González,
Rafael Suárez-Vega,
María Isabel Dorta-González
Abstract:
There are many factors that affect the probability of being uncited during the first years after publication. In this study, we analyze three of these factors for journals, conference proceedings and book series: the field (in 316 subject categories of the Scopus database), the access modality (open access vs. paywalled), and the visibility of the source (through the percentile of the average impa…
▽ More
There are many factors that affect the probability of being uncited during the first years after publication. In this study, we analyze three of these factors for journals, conference proceedings and book series: the field (in 316 subject categories of the Scopus database), the access modality (open access vs. paywalled), and the visibility of the source (through the percentile of the average impact in the subject category). We quantify the effect of these factors on the probability of being uncited. This probability is measured through the percentage of uncited documents in the serial sources of the Scopus database at about two years after publication. As a main result, we do not find any strong correlation between open access and uncitedness. Within the group of most cited journals (Q1 and top 10%), open access journals generally have somewhat lower uncited rates. However, in the intermediate quartiles (Q2 and Q3) almost no differences are observed, while for Q4 the uncited rate is again somewhat lower in the case of the OA group. This is important because it provides new evidence in the debate about open access citation advantage.
△ Less
Submitted 19 May, 2020;
originally announced May 2020.
-
Interdisciplinarity metric based on the co-citation network
Authors:
Juan María Hernández,
Pablo Dorta-González
Abstract:
Quantifying the interdisciplinarity of a research is a relevant problem in the evaluative bibliometrics. The concept of interdisciplinarity is ambiguous and multidimensional. Thus, different measures of interdisciplinarity have been propose in the literature. However, few studies have proposed interdisciplinary metrics without previously defining classification sets, and no one use the co-citation…
▽ More
Quantifying the interdisciplinarity of a research is a relevant problem in the evaluative bibliometrics. The concept of interdisciplinarity is ambiguous and multidimensional. Thus, different measures of interdisciplinarity have been propose in the literature. However, few studies have proposed interdisciplinary metrics without previously defining classification sets, and no one use the co-citation network for this purpose. In this study we propose an interdisciplinary metric based on the co-citation network. This is a way to define the publication's field without resorting to pre-defined classification sets. We present a characterization of a publication's field and then we use this definition to propose a new metric of the interdisciplinarity degree for publications (papers) and journals as units of analysis. The proposed measure has an aggregative property that makes it scalable from a paper individually to a set of them (journal) without more than adding the numerators and denominators in the proportions that define this new indicator. Moreover, the aggregated value of two or more units is strictly among all the individual values.
△ Less
Submitted 23 March, 2020;
originally announced March 2020.
-
Publication modalities 'article in press' and 'open access' in relation to journal average citation
Authors:
Sara M. González-Betancor,
Pablo Dorta-González
Abstract:
There has been a generalization in the use of two publication practices by scientific journals during the past decade: 1. 'article in press' or early view, which allows access to the accepted paper before its formal publication in an issue; 2. 'open access', which allows readers to obtain it freely and free of charge. This paper studies the influence of both publication modalities on the average i…
▽ More
There has been a generalization in the use of two publication practices by scientific journals during the past decade: 1. 'article in press' or early view, which allows access to the accepted paper before its formal publication in an issue; 2. 'open access', which allows readers to obtain it freely and free of charge. This paper studies the influence of both publication modalities on the average impact of the journal and its evolution over time. It tries to identify the separate effect of access on citation into two major parts: early view and selection effect, managing to provide some evidence of the positive effect of both. Scopus is used as the database and CiteScore as the measure of journal impact. The prevalence of both publication modalities is quantified. Differences in the average impact factor of group of journals, according to their publication modalities, are tested. The evolution over time of the citation influence, from 2011 to 2016, is also analysed. Finally, a linear regression to explain the correlation of these publication practices with the CiteScore in 2016, in a ceteris paribus context, is estimated. Our main findings show evidence of a positive correlation between average journal impact and advancing the publication of accepted articles, moreover this correlation increases over time. The open access modality, in a ceteris paribus context, also correlates positively with average journal impact.
△ Less
Submitted 8 July, 2019;
originally announced July 2019.
-
Characterizing the highly cited articles: a large-scale bibliometric analysis of the top 1% most cited research
Authors:
Pablo Dorta-González,
Yolanda Santana-Jiménez
Abstract:
We conducted a large-scale analysis of around 10,000 scientific articles, from the period 2007-2016, to study the bibliometric or formal aspects influencing citations. A transversal analysis was conducted disaggregating the articles into more than one hundred scientific areas and two groups, one experimental and one control, each with a random sample of around five thousand documents. The experime…
▽ More
We conducted a large-scale analysis of around 10,000 scientific articles, from the period 2007-2016, to study the bibliometric or formal aspects influencing citations. A transversal analysis was conducted disaggregating the articles into more than one hundred scientific areas and two groups, one experimental and one control, each with a random sample of around five thousand documents. The experimental group comprised a random sample of the top 1% most cited articles in each field and year of publication (highly cited articles), and the control group a random sample of the remaining articles in the Journal Citation Reports (science and social science citation indexes in the Web of Science database). As the main result, highly cited articles differ from non-highly cited articles in most of the bibliometric aspects considered. There are significant differences, below the 0.01 level, between the groups of articles in many variables and areas. The highly cited articles are published in journals of higher impact factor (33 percentile points above) and have 25% higher co-authorship. The highly cited articles are also longer in terms of number of pages (10% higher) and bibliographical references (35% more). Finally, highly cited articles have slightly shorter titles (3% lower) but, contrastingly, longer abstracts (10% higher).
△ Less
Submitted 27 April, 2018;
originally announced April 2018.
-
Prevalence and citation advantage of gold open access in the subject areas of the Scopus database
Authors:
Pablo Dorta-González,
Yolanda Santana-Jiménez
Abstract:
The potential benefit of open access (OA) in relation to citation impact has been discussed in the literature in depth. The methodology used to test the OA citation advantage includes comparing OA vs. non-OA journal impact factors and citations of OA versus non-OA articles published in the same non-OA journals. However, one problem with many studies is that they are small -restricted to a discipli…
▽ More
The potential benefit of open access (OA) in relation to citation impact has been discussed in the literature in depth. The methodology used to test the OA citation advantage includes comparing OA vs. non-OA journal impact factors and citations of OA versus non-OA articles published in the same non-OA journals. However, one problem with many studies is that they are small -restricted to a discipline or set of journals-. Moreover, conclusions are not entirely consistent among research areas and 'early view' and 'selection bias' have been suggested as possible explications. In the present paper, an analysis of gold OA from across all areas of research -the 27 subject areas of the Scopus database- is realized. As a novel contribution, this paper takes a journal-level approach to assessing the OA citation advantage, whereas many others take a paper-level approach. Data were obtained from Scimago Lab, sorted using Scopus database, and tagged as OA/non-OA using the DOAJ list. Jointly with the OA citation advantage, the OA prevalence as well as the differences between access types (OA vs. non-OA) in production and referencing are tested. A total of 3,737 OA journals (16.8%) and 18,485 non-OA journals (83.2%) published in 2015 are considered. As the main conclusion, there is no generalizable gold OA citation advantage at journal level.
△ Less
Submitted 11 September, 2017; v1 submitted 21 August, 2017;
originally announced August 2017.
-
Reconsidering the gold open access citation advantage postulate in a multidisciplinary context: an analysis of the subject categories in the Web of Science database 2009-2014
Authors:
Pablo Dorta-Gonzalez,
Sara M. Gonzalez-Betancor,
Maria Isabel Dorta-Gonzalez
Abstract:
Since Lawrence in 2001 proposed the open access (OA) citation advantage, the potential benefit of OA in relation to the citation impact has been discussed in depth. The methodology to test this postulate ranges from comparing the impact factors of OA journals versus traditional ones, to comparing citations of OA versus non-OA articles published in the same non-OA journals. However, conclusions are…
▽ More
Since Lawrence in 2001 proposed the open access (OA) citation advantage, the potential benefit of OA in relation to the citation impact has been discussed in depth. The methodology to test this postulate ranges from comparing the impact factors of OA journals versus traditional ones, to comparing citations of OA versus non-OA articles published in the same non-OA journals. However, conclusions are not entirely consistent among fields, and two possible explications have been suggested in those fields where a citation advantage has been observed for OA: the early view and the selection bias postulates. In this study, a longitudinal and multidisciplinary analysis of the gold OA citation advantage is developed. All research articles in all journals for all subject categories in the multidisciplinary database Web of Science are considered. A total of 1,137,634 articles - 86,712 OA articles (7.6%) and 1,050,922 non-OA articles (92.4%)- published in 2009 are analysed. The citation window considered goes from 2009 to 2014, and data are aggregated for the 249 disciplines (subject categories). At journal level, we also study the evolution of journal impact factors for OA and non-OA journals in those disciplines whose OA prevalence is higher (top 36 subject categories). As the main conclusion, there is no generalizable gold OA citation advantage, neither at article nor at journal level.
△ Less
Submitted 9 March, 2017;
originally announced March 2017.
-
An indicator of journal impact that is based on calculating a journal's percentage of highly cited publications
Authors:
Sara M. Gonzalez-Betancor,
Pablo Dorta-Gonzalez
Abstract:
The two most used citation impact indicators in the assessment of scientific journals are, nowadays, the impact factor and the h-index. However, both indicators are not field normalized (vary heavily depending on the scientific category) which makes them incomparable between categories. Furthermore, the impact factor is not robust to the presence of articles with a large number of citations, while…
▽ More
The two most used citation impact indicators in the assessment of scientific journals are, nowadays, the impact factor and the h-index. However, both indicators are not field normalized (vary heavily depending on the scientific category) which makes them incomparable between categories. Furthermore, the impact factor is not robust to the presence of articles with a large number of citations, while the h-index depends on the journal size. These limitations are very important when comparing journals of different sizes and categories. An alternative citation impact indicator is the percentage of highly cited articles in a journal. This measure is field normalized (comparable between scientific categories), independent of the journal size and also robust to the presence of articles with a high number of citations. This paper empirically compares this indicator with the impact factor and the h-index, considering different time windows and citation percentiles (levels of citation for considering an article as highly cited compared to others in the same year and category).
△ Less
Submitted 13 October, 2015;
originally announced October 2015.
-
Research status and trends in Operations Research and Management Science (OR/MS) journals: A bibliometric analysis based on the Web of Science database 2001-2012
Authors:
Pablo Dorta-Gonzalez,
Maria Isabel Dorta-Gonzalez,
Dolores Rosa Santos-Penate,
Rafael Suarez-Vega
Abstract:
A bibliometric analysis to evaluate global scientific production in the subject category of Operations Research and Management Science (OR/MS) from 2001 to 2012 was applied. Data was based on the Web of Science (Science Citation Index) database compiled by Thomson Reuters. The results showed that the OR/MS research has significantly increased over the past twelve years. The Bradford core journals…
▽ More
A bibliometric analysis to evaluate global scientific production in the subject category of Operations Research and Management Science (OR/MS) from 2001 to 2012 was applied. Data was based on the Web of Science (Science Citation Index) database compiled by Thomson Reuters. The results showed that the OR/MS research has significantly increased over the past twelve years. The Bradford core journals in the category were identified. The researchers paid great attention to networks, control, and simulation. Among the countries, USA attained a dominant position in global research in the field.
△ Less
Submitted 17 October, 2014;
originally announced October 2014.
-
An approach to the author citation potential: Measures of scientific performance which are invariant across scientific fields
Authors:
Pablo Dorta-Gonzalez,
Maria Isabel Dorta-Gonzalez,
Rafael Suarez-Vega
Abstract:
The citation potential is a measure of the probability of being cited. Obviously, it is different among fields of science, social science, and humanities because of systematic differences in publication and citation behaviour across disciplines. In the past, the citation potential was studied at journal level considering the average number of references in established groups of journals (for examp…
▽ More
The citation potential is a measure of the probability of being cited. Obviously, it is different among fields of science, social science, and humanities because of systematic differences in publication and citation behaviour across disciplines. In the past, the citation potential was studied at journal level considering the average number of references in established groups of journals (for example, the crown indicator is based on the journal subject categories in the Web of Science database). In this paper, some characterizations of the author's scientific research through three different research dimensions are proposed: production (journal papers), impact (journal citations), and reference (bibliographical sources). Then, we propose different measures of the citation potential for authors based on a proportion of these dimensions. An empirical application, in a set of 120 randomly selected highly productive authors from the CSIC Research Centre (Spain) in four subject areas, shows that the ratio between production and impact dimensions is a normalized measure of the citation potential at the level of individual authors. Moreover, this ratio reduces the between-group variance in relation to the within-group variance in a higher proportion than the rest of the indicators analysed. Furthermore, it is consistent with the type of journal impact indicator used. A possible application of this result is in the selection and promotion process within interdisciplinary institutions, since it allows comparisons of authors based on their particular scientific research.
△ Less
Submitted 8 October, 2014;
originally announced October 2014.
-
Journal topic citation potential and between-field comparisons: The topic normalized impact factor
Authors:
Pablo Dorta-Gonzalez,
Maria Isabel Dorta-Gonzalez,
Dolores Rosa Santos-Penate,
Rafael Suarez-Vega
Abstract:
The journal impact factor is not comparable among fields of science and social science because of systematic differences in publication and citation behaviour across disciplines. In this work, a source normalization of the journal impact factor is proposed. We use the aggregate impact factor of the citing journals as a measure of the citation potential in the journal topic, and we employ this cita…
▽ More
The journal impact factor is not comparable among fields of science and social science because of systematic differences in publication and citation behaviour across disciplines. In this work, a source normalization of the journal impact factor is proposed. We use the aggregate impact factor of the citing journals as a measure of the citation potential in the journal topic, and we employ this citation potential in the normalization of the journal impact factor to make it comparable between scientific fields. An empirical application comparing some impact indicators with our topic normalized impact factor in a set of 224 journals from four different fields shows that our normalization, using the citation potential in the journal topic, reduces the between-group variance with respect to the within-group variance in a higher proportion than the rest of indicators analysed. The effect of journal self-citations over the normalization process is also studied.
△ Less
Submitted 25 February, 2014;
originally announced February 2014.
-
Central indexes to the citation distribution: A complement to the h-index
Authors:
Pablo Dorta-Gonzalez,
Maria Isabel Dorta-Gonzalez
Abstract:
The citation distribution of a researcher shows the impact of their production and determines the success of their scientific career. However, its application in scientific evaluation is difficult due to the bi-dimensional character of the distribution. Some bibliometric indexes that try to synthesize in a numerical value the principal characteristics of this distribution have been proposed recent…
▽ More
The citation distribution of a researcher shows the impact of their production and determines the success of their scientific career. However, its application in scientific evaluation is difficult due to the bi-dimensional character of the distribution. Some bibliometric indexes that try to synthesize in a numerical value the principal characteristics of this distribution have been proposed recently. In contrast with other bibliometric measures, the biases that the distribution tails provoke, are reduced by the h-index. However, some limitations in the discrimination among researchers with different publication habits are presented in this index. This index penalizes selective researchers, distinguished by the large number of citations received, as compared to large producers. In this work, two original sets of indexes, the central area indexes and the central interval indexes, that complement the h-index to include the central shape of the citation distribution, are proposed and compared.
△ Less
Submitted 18 April, 2013;
originally announced April 2013.
-
Comparing journals from different fields of Science and Social Science through a JCR Subject Categories Normalized Impact Factor
Authors:
Pablo Dorta-Gonzalez,
Maria Isabel Dorta-Gonzalez
Abstract:
The journal Impact Factor (IF) is not comparable among fields of Science and Social Science because of systematic differences in publication and citation behaviour across disciplines. In this work, a decomposing of the field aggregate impact factor into five normally distributed variables is presented. Considering these factors, a Principal Component Analysis is employed to find the sources of the…
▽ More
The journal Impact Factor (IF) is not comparable among fields of Science and Social Science because of systematic differences in publication and citation behaviour across disciplines. In this work, a decomposing of the field aggregate impact factor into five normally distributed variables is presented. Considering these factors, a Principal Component Analysis is employed to find the sources of the variance in the JCR subject categories of Science and Social Science. Although publication and citation behaviour differs largely across disciplines, principal components explain more than 78% of the total variance and the average number of references per paper is not the primary factor explaining the variance in impact factors across categories. The Categories Normalized Impact Factor (CNIF) based on the JCR subject category list is proposed and compared with the IF. This normalization is achieved by considering all the indexing categories of each journal. An empirical application, with one hundred journals in two or more subject categories of economics and business, shows that the gap between rankings is reduced around 32% in the journals analyzed. This gap is obtained as the maximum distance among the ranking percentiles from all categories where each journal is included.
△ Less
Submitted 18 April, 2013;
originally announced April 2013.
-
Impact maturity times and citation time windows: The 2-year maximum journal impact factor
Authors:
Pablo Dorta-Gonzalez,
Maria Isabel Dorta-Gonzalez
Abstract:
Journal metrics are employed for the assessment of scientific scholar journals from a general bibliometric perspective. In this context, the Thomson Reuters journal impact factors (JIF) are the citation-based indicators most used. The 2-year journal impact factor (2-JIF) counts citations to one and two year old articles, while the 5-year journal impact factor (5-JIF) counts citations from one to f…
▽ More
Journal metrics are employed for the assessment of scientific scholar journals from a general bibliometric perspective. In this context, the Thomson Reuters journal impact factors (JIF) are the citation-based indicators most used. The 2-year journal impact factor (2-JIF) counts citations to one and two year old articles, while the 5-year journal impact factor (5-JIF) counts citations from one to five year old articles. Nevertheless, these indicators are not comparable among fields of science for two reasons: (i) each field has a different impact maturity time, and (ii) because of systematic differences in publication and citation behaviour across disciplines. In fact, the 5-JIF firstly appeared in the Journal Citation Reports (JCR) in 2007 with the purpose of making more comparable impacts in fields in which impact matures slowly. However, there is not an optimal fixed impact maturity time valid for all the fields. In some of them two years provides a good performance whereas in others three or more years are necessary. Therefore, there is a problem when comparing a journal from a field in which impact matures slowly with a journal from a field in which impact matures rapidly. In this work, we propose the 2-year maximum journal impact factor (2M-JIF), a new impact indicator that considers the 2-year rolling citation time window of maximum impact instead of the previous 2-year time window. Finally, an empirical application comparing 2-JIF, 5-JIF, and 2M-JIF shows that the maximum rolling target window reduces the between-group variance with respect to the within-group variance in a random sample of about six hundred journals from eight different fields.
△ Less
Submitted 18 April, 2013;
originally announced April 2013.
