KEY TAKEAWAYS

• The ‘diversity factor’ has been proposed as a new, more equitable metric for assessing journal quality and the impact of health research.
• The index takes into account the diversity of the authors, study participants, and departmental affiliations to promote a wider range of perspectives in research.

The impact factor remains the dominant metric among researchers for assessing journal and (indirectly) research paper quality, despite multiple calls for it to be superseded by alternative measures. Recently, a novel metric claimed the spotlight in an MIT News article. The article describes a study by Dr Jack Gallifant et al., published in PLOS Global Public Health, which suggests that the impact factor misses the mark in capturing a paper’s impact on health. The researchers argue that, for a more accurate understanding of impact, journal metrics should take into account the diversity of the authors and of the study participants. They propose a novel metric, termed the ‘diversity factor’.

The index is comprised of 3 key components:

• author demographics: the gender and geographic location of the authors
• participant demographics: the gender, ethnicity, race, language, geographic location, and age of the individuals enrolled in the study
• departmental affiliation: papers with authors from different disciplines (eg, doctors, nurses, and engineers) score more highly than papers with authors from a single field.

After settling on the metric’s components, the group used the database OpenAlex to extract metadata relating to the authors of over 100,000 medical papers, from around 7,500 journals, published in the last 20 years. A considerable number of the papers retrieved were not open access, meaning that participant demographics could not be included in the final analysis. However, as the researchers predicted, most papers did not perform well against the new metric, even when considering author information alone. Specifically, there was significant underrepresentation of female authors and of authors from low- or middle-income countries. The group hope that by quantifying and tracking diversity in this way, over time, those working in health research would be prompted to drive progress against these measures.

So, why exactly is a lack of diversity a problem for global health outcomes? Ultimately, it boils down to ‘blind spots’ in medical knowledge, explains Dr Leo Anthony Celi, senior author of the paper:

“What happens when all of the authors involved in a project are alike is that they’re going to have the same blind spots. They’re all going to see the problem from the same angle. What we need is cognitive diversity, which is predicated on lived experiences.”

Dr Celi believes that stakeholders within medical publishing — including journals, academic institutions, funding bodies, and even the media — are accountable for the inequity seen in health research. As such, each must play their part in diversifying medical research publications. To this end, Dr Celi calls for the diversity factor to prompt discussions within the medical research community and provide a first step towards a more equitable evaluation of the true impact of research.

————————————————