Guides: Assessing Research Impact: The Debate over Metrics: Readings

The Debate over Metrics: Selected Readings

There is much debate over the validity of metrics to measure the value of individual articles, author output, journals, and institutions. Below is a selected bibliography of some of the papers, statements, policies, and manifestos that have been published on the topic.

The Metric Tide
Wilsdon, J., et al. (2015). The Metric Tide: Report of the Independent Review of the Role of
Metrics in Research Assessment and Management. DOI: 10.13140/RG.2.1.4929.1363
Altmetrics: a manifesto
Counting what counts: The debate over citation metrics
A new wave of data-driven research tools is reigniting an age-old conversation around what metrics can be used... and abused... when statistically analysing scientific literature quality.
The h-index is no longer an effective correlate of scientific reputation
Koltun V, Hafner D (2021) The h-index is no longer an effective correlate of scientific reputation. PLoS ONE 16(6): e0253397. https://doi.org/10.1371/journal.pone.0253397
The impact of individual scientists is commonly quantified using citation-based measures. The most common such measure is the h-index. A scientist’s h-index affects hiring, promotion, and funding decisions, and thus shapes the progress of science. The correlation of the h-index with awards that indicate recognition by the scientific community has substantially declined.

Principles of Responsible Metrics Use: Selected Readings

Leiden Manifesto for Research Metrics Proposes 10 principles for the measurement of research performance; "distillation of best practice in metrics-based research assessment"
San Francisco Declaration on Research Assessment (DORA) Excerpt: "The outputs from scientific research are many and varied, including: research articles reporting new knowledge, data, reagents, and software; intellectual property; and highly trained young scientists. Funding agencies, institutions that employ scientists, and scientists themselves, all have a desire, and need, to assess the quality and impact of scientific outputs. It is thus imperative that scientific output is measured accurately and evaluated wisely..."
Hong Kong Principles for Assessing Researchers "We have developed the Hong Kong Principles (HKPs)...with a specific focus on the need to drive research improvement through ensuring that researchers are explicitly recognized and rewarded for behaviors that strengthen research integrity. We present five principles: responsible research practices; transparent reporting; open science (open research); valuing a diversity of types of research; and recognizing all contributions to research and scholarly activity."
Helsinki Initiative on Multilingualism in Scholarly Communication Research is international. That's the way we like it! Multilingualism keeps locally relevant research alive. Protect it! Disseminating research results in your own language creates impact. Endorse it! It is vital to interact with society and share knowledge beyond academia. Promote it! Infrastructure of scholarly communication in national languages is fragile. Don't lose it!
Principles of Research Assessment and Management from the University of Bath "The principles that outline our approach to research assessment and management, including the responsible use of quantitative indicators." (University of Bath)
Research Evaluation Working Group of INORMS The aim of this working group is to develop a joint programme of work and share experiences and best practice with a view to ensuring research evaluation is meaningful, responsible and effective. The working group is a part of INORMS; INORMS was formed in 2001 to bring together research management societies and associations from across the globe...to enable interactions, sharing of good practice, and joint activities between the member societies.
SCOPE Process for Evaluating Responsibly