Metrics, sometimes known as research indicators, are intended to be a quantitative measure of the quality and/or importance of a research output (e.g. journal article, monograph). There is a range of metrics that measure different things.
Citation metrics
- They measure different things and it is important to understand what they measure, the limits of what they interpret, and when best to use them.
- They are sometimes used to measure the citation impact of journals rather than the citation impact of individual research outputs.
- They measure the amount of academic attention a research output has received.
- The simplest citation metric is the number of times a research output has been cited.
- Citation practices differ across academic disciplines so there are a variety of weightings and calculations to address that and ensure that citation metrics are comparable.
The Journal Impact Factor (JIF) is a widely used example of a citation metric. It calculates the average number of citations that articles published by a journal have received over the preceding two-year period. The JIF and other metrics are problematic for a number of reasons. One cannot judge the academic or research value of a work merely on the number of times it has been downloaded, or the number of times it has been cited.
Journal-level metrics are often used by researchers to make decisions about where to publish their work. They can be useful for identifying journals that have a track record for generating attention for the research outputs that they publish.
Alternative metrics or altmetrics measure the amount of online attention a research output has received beyond typical citation metrics. This can include things like mentions on social media, within policy documents, in coverage on news platforms and blogs, Wikipedia citations and lots more besides. Measuring this kind of attention can be used to demonstrate reach beyond academia, and can help to evidence research impact.
Before deciding upon which metrics you might use for your research evaluation, it is worthwhile to consider what you are trying to measure and how you will do so. Inorms have a step-by-step guide to assist with research evaluations.
Responsible Metrics
There has been a movement towards advocating for responsible metrics which provides guidance on how to use metrics responsibly, both for measuring research quality and as part of conversations about career progression. There are a number of different declarations and agreements on research assessment. The San Francisco Declaration on Research Assessment (DORA) is a set of principles that institutions have signed up to, which includes a requirement to eliminate the use of journal-level metrics (like the Journal Impact Factor) in funding, appointment and promotion considerations. The Coalition for Advancing Research Assessment is a collective of organisations that are either signatories or members, committed to reform research assessment. In response, some universities have created statements of responsible metrics. The Bibliomagician has collated a number of examples from a variety of institutions.