The 8^th European Medical Writers Association (EMWA) symposium, entitled ‘Research Integrity & The Medical Communicator: What We Do When No One Is Watching’ took place on 6^th November. Researchers, journal publishers and representatives from the pharmaceutical industry and medical communication agencies joined together in this virtual event, to share their perspectives on the importance of research integrity and how it can be achieved.

A summary of session 1 of the meeting is provided below to benefit those who were unable to attend, and as a timely reminder of the key topics for those who did.

Read our summaries of session 2 and session 3 of the meeting.

Welcome and Introduction

Slávka Baróniková, Conference Director of EMWA and Co-Chair of the Medical Communications Special Interest Group (SIG) of EMWA, opened the meeting and introduced the Co-Chairs, Chris Winchester (Oxford PharmaGenesis) and Martin Delahunty (Inspiring STEM Consulting). Delahunty gave a flavour of the topics that would be covered in the symposium by outlining some contextual factors for consideration. He noted that the real problem within medical publications is not retractions or fraud but a lack of transparency, reproducibility and replicability. Concerns around research integrity and peer review have been brought to the fore by the Surgisphere scandal amid the COVID-19 pandemic, but this has only amplified a long-standing problem. In essence, we still struggle with the issue of research integrity and peer review.

Winchester continued to note that, clearly, these issues affect the whole of science, but focusing on the pharmaceutical industry specifically, with its unique set of challenges, it is worrying that less than half of the UK general public trust research conducted by pharmaceutical or industry scientists. While the pharmaceutical industry has previously been criticised, things are improving. It’s good to see, for example, pharma coming together through the Open Pharma movement, demonstrating proactivity towards communicating research in a more transparent, accessible, timely and sustainable way.

Winchester concluded his introduction by outlining the three main topics that would be explored during the day:

1. Why research integrity matters to all of us
2. How we can strengthen research integrity
3. Putting it all into practice – what we can do as medical communicators working with other stakeholders to improve research integrity

Session 1: why research integrity matters to all of us

Research reproducibility and the need for Open Science

 Sandra Petty (University of Melbourne; Center for Biomedical Research Transparency [CBMRT]) began the first presentation of the meeting by stating that while it is important to recognise the many great discoveries in science, a number of areas need to be addressed to improve trust in science and biomedical clinical research, including:

• reproducibility rates
• biases
• statistical methods
• the peer review process
• accountability
• key performance indicators driving behaviours
• access to data and information
• predatory journals

Petty noted that 85% of health research is wasted (because completed research is not published; is not sufficiently clear, complete, and accurate for others to interpret, use, or replicate the research correctly; or does not take systematic account of lessons and results from previous, related research), with an implied annual cost of $170 billion. Notwithstanding the need for further analysis of this finding, it remains clear that we should stop and think about the quality of the research we are conducting or writing up.

Not only does research irreproducibility have substantial financial costs, it also has implications for patient safety. In a widely reported analysis, Amgen found that only 11% (6/53) of published preclinical study findings in haematology/oncology could be confirmed. Conducting clinical trials based on non-replicable preclinical study findings may ultimately lead to a higher likelihood of patients enrolling in ineffective (and potentially harmful) trials.

Research practices were the focus of much attention and debate following the publication of an essay in PLoS Medicine in 2005 which has since been viewed almost 3.2 million times: ‘Why Most Published Research Findings Are False’. Inadequate statistical methods, bias, ‘flexible research design’, financial associations and ‘hot topics’ were all highlighted as factors contributing to the finding that the vast majority of published research claims are false. The publication serves as an important reminder of the need for good research practice.

More recently, the authors of a 2014 publication estimated the rate of false discovery among major medical journals to be 14%. It found no significant increase in the rate over time and concluded that “statistical analysis must allow for false discoveries in order to make claims on the basis of noisy data” and “the medical literature remains a reliable record of scientific progress”. Petty argued, however, that many more positive than negative findings have been published over time, creating publication bias in the medical literature. Petty highlighted the need to keep coming back to and correcting the scientific record in line with updates, errors and retractions to ensure its accuracy is maintained.

In an opinion piece in PNAS from 2018, Daniele Fanelli argued that a narrative of ‘scientific crisis’ exists within the literature but this may be counterproductive in promoting better science. Petty concurred that a scientific crisis narrative may shift the focus away from the real need to address the problems in research. How then, asked Petty, can we create a better opportunity for transparency and reproducibility?

Multiple stakeholders are involved in, and have the opportunity to enhance, the research cycle by making it more transparent and reproducible. Having access to the data, writers and authors have a vital role in ensuring transparency.

Petty noted the following tools available for improving rigour, transparency and discoverability, although acknowledged that many of these require more refinement and ‘loop closure’:

• clinical trial registries
• preclinical research registries
• preregistration of studies (methods, statistics and endpoints)
• data sharing (with appropriate de-identification)
• addressing publication bias and delays
• preprints (as they have not yet been peer-reviewed, they should be treated as preliminary, and so can cause issues when they are picked up by the mainstream media)
• open access.

Petty highlighted the numerous potential benefits of open science: enhancing collaboration, speeding up discovery, facilitating feedback at an earlier stage, increasing access to null/replicative studies and encouraging data sharing. From an open access publishing perspective, the Plan S initiative, requiring that scientific publications from research funded by public grants must be published in open access journals or platforms, will apply to research grant applications from 1 January 2021.

Another consideration is the current culture of scientific research. Academic benchmarks and key performance indicators (KPIs) need to be revised so that rigour, reproducibility, transparency and collaboration are rewarded. Importantly, journal impact factors are not intended to validate and score an individual paper. In addition, so-called ‘negative papers’ should not automatically be negatively received; according to the CBMRT ‘a result is a result’ and the quality of research, rather than the direction of the result, should be the focus. This is the focus of one of CBMRT’s projects: the Null Hypothesis Initiative. CBMRT are partnering with Neurology to include results from well-performed negative studies in the main journal. One of the successes of the initiative was the publication of a study demonstrating that fluoxetine, which had been used off-label in the treatment of EV-D68 in acute flaccid myelitis, was tolerable but not efficacious. The publication achieved high Altmetric scores and was picked up in the mainstream media. Publishing negative results is helpful for clinicians, the research field and funders. CBMRT are now working with the American Heart Association to create a Null Hypothesis Collection across their journals, which is another step towards this becoming a movement across every major field, a key target for CBMRT.

Petty concluded that while challenges exist in a competitive environment and funding is needed to increase uptake, open medical research has the potential to correct much of the problem of research reproducibility.

The dark secret linking research integrity and publication ethics

Simon Linacre (Cabells) focused on the vast and concerning issue of predatory publishing, by providing a brief history of the problem, explaining the specific threats and implications, describing how predatory publishers have responded to COVID-19 and outlining ways in which predatory publishing can be combated.

Although research integrity and publication ethics are often considered as separate entities, there is considerable overlap. Changes in the publishing environment, including the increase in speed and options for publication, including open access and preprint servers, provide the opportunity to publish quickly and seamlessly.

Predatory publishing is at the nexus of problems around these changes in the publishing model and affects all stakeholders – researchers, funders, universities, and publishers – with a net effect of damaged trust in academic research.

The term ‘predatory journals’ was first coined in 2008, with the publication of the now-defunct Beall’s List. Since then, the situation has continued to worsen, and Cabells’ ‘Black List’ (renamed Predatory Reports) now includes approximately 14,000 predatory journals.

One of the main threats posed by predatory publishing is the minimal, or sometimes, complete absence of, peer review, which results in the publication of unchecked research. Such activity can be associated with significant financial cost. In 2019, the US Federal Trade Commission judged against the largest predatory journal operator (OMICS Group Inc. based in India). They estimated that OMICS had defrauded authors who had published in their 700+ journals by more than $50 million over 6 years. ‘Back of the envelope’ calculations by Cabells estimate that $100 million is spent annually by authors publishing in these journals.

In an effort to increase awareness of the problem, definitions of predatory journals have been developed by a number of organisations over recent years. A group of 43 stakeholders published the following definition in Nature in 2019: ‘Predatory journals and publishers are entities that prioritise self-interest at the expense of scholarship and are characterised by false or misleading information, deviation from best editorial and publication practices, a lack of transparency, and/or the use of aggressive and indiscriminate solicitation practices.’ Linacre felt that while this encapsulates the problem of predatory publishing, it also raises some questions, for example, how do you know if a publisher is prioritising self-interest? In addition, this definition does not cover peer review. Position statements that do include inadequate or non-existent peer review in their definition of predatory journals have been developed by COPE and by AMWA-EMWA-ISMPP. Definitions of predatory publishing are valuable in increasing awareness of the practice to authors when they approach the decision of where to submit their research.

Some scepticism exists with regards to the extent and implications of the problem. Researchers may believe that they would spot a predatory publisher. However, Linacre described how a professor from a major US institution had been duped into speaking at a predatory congress. “It may be easier than you think to get sucked in,” Linacre said.

So why should you care about predatory publishing? Linacre described five compelling reasons:

1. Without peer review, medical content is either unchecked, unusable or undiscoverable – good research that’s worth sharing can’t be found as it won’t appear in publication indexes.
2. Predatory publishers exploit the open access business model for profit – authors are paying to be published but not getting the service they’d expect.
3. Research integrity is not a priority – often linked to criminal activities/gangs.
4. Researchers are deceived/defrauded and taken advantage of due to ‘publish or perish’ scenarios they face. Unless there is a cultural change then this temptation will remain, perpetuating the problem.
5. Trust in medical research is eroded – medicine is an area of particular focus for predatory publishers (as it receives more funding via article processing charges [APCs] than other academic disciplines).

Perhaps unsurprisingly, the current focus on COVID-19 has prompted predatory publishers to target this area of research. Ongoing research by Cabells has found that 100+ predatory publications have come out of tier one universities in one country since the start of 2019 and 20% of these are COVID-related. The predatory publishing environment is also changing its business model in response to the COVID-19 situation, for example, by moving their congresses online.

The problem of predatory publishing is not restricted to academia. In 2018, a German news consortium found that more than half of Germany’s top 30 companies had employees who had either published articles in such journals or participated in their conferences. Large pharmaceutical companies were among the organisations that had published in such journals, leaving them open to the accusation that they have deliberately chosen an outlet for easily publishing research without going through a proper peer review process.

In conclusion, Linacre outlined what can be done to combat ‘the dark secret’ and thus protect all stakeholders in the industry:

• remove the motivations to submit to predatory journals/conferences (but the culture of ‘publish or perish’ is embedded and will take time to change)
• improve education of researchers
• reprove protagonists – ensure whistleblowers are protected
• use technology (eg using IP addresses of known predatory publishing organisations to flag access to their websites)
• approve sources – citing credible sources is key.

Research integrity – the pharmaceutical company perspective

Anna-Lisa Fisher (Boehringer Ingelheim) noted that publications represent the tangible results of the huge investments made by pharma in medical research and are a lasting legacy of advances in healthcare. Pharma publications keep healthcare professionals informed and thus ensure patients have the best possible treatment, but how are they perceived by:

• The general public? Data from the UK general public showed that 59% of respondents trust university scientists and only 32% trust scientists from pharma companies. Reasons for a lack of trust included concerns around corruption, exaggeration and dishonesty.
• Scientists? Only 15% of scientists in a worldwide survey felt that the public trust the scientific community and worryingly about half of scientists had awareness themselves of scientific misconduct. Compared with academic scientists, scientists in industry were much more likely to report negative consequences of scientific conduct.

Fisher asked: can we find a correlation between the number of retractions in academia versus pharma?

• A retrospective analysis found that publications retracted due to misconduct rarely involve pharma or medical writing support.
• Only 3.4% of retractions owing to misconduct involved the pharmaceutical industry and this rate was reduced to zero when medical writers were also involved.

To recap, the level of public trust in pharma is about 30% even though they are involved with less than 4% of all known retractions due to misconduct, whereas the level of trust in academia is about 60% with academia research accounting for 96% of all retractions.

Fisher discussed the awareness and enforcement of publications guidelines among academia vs pharma, referring to two surveys:

• A survey published in BMJ Open of almost 4,000 active researchers who were predominantly from universities showed:
  • 74% of responders were familiar with International Committee of Medical Journal Editor (ICMJE) authorship criteria
  • only 57% reported that they had used these to decide authorship on their last paper (increasing to 64% among those who were aware of guidelines)
  • only 37% of respondents had an institutional authorship policy.
• A 2013 survey by Wager et al. of 469 publication professionals published in BMJ Open found:
  • almost universal awareness of international guidelines on responsible publication practice
  • more than 90% of industry/agency and contract research organisation respondents routinely use GPP2 and ICMJE guidelines
  • most received mandatory training on policies/SOPs and medical writer support was almost universally acknowledged in publications
  • approximately half of industry respondents have full internal audits to ensure compliance; fewer agency individuals reported that they were audited (20%)
  • 37% of agency respondents reported being asked by an author or a sponsor to do something that they believed was unethical but, encouragingly, the majority were withdrawn once the need for guideline compliance was explained
  • most respondents had been involved in publishing studies with negative or inconclusive results.

Fisher noted that in light of the huge amount of information that needs to be read and understood to ensure compliance, transparency guidelines need to be ‘baked in’ to the publication process. Most pharma companies have an internal transparency policy that underpins their publications SOP and there are usually also local working instructions to address specific local rules and regulations. These form the guidelines by which most pharma develop and share their data through publications. Most companies also provide comprehensive mandatory training for those involved in publications. Embedding of compliance in the day job is also achieved with the use of dedicated tools to plan, manage and track publications and store publication reviews and approvals.

Fisher concluded that:

• pharma is less likely to be involved with several measures of scientific misconduct because of the huge effort that has gone into ‘baking in’ research integrity to day-to-day processes practices
• trust in pharma remains low, however, because public perception isn’t always based on fact and it can take decades to rebuild lost trust
• if we don’t learn from the past and know why the guidelines came about, history could be repeated, which could be a deadly blow for pharma
• potential conflict between research integrity and the bottom line is always lurking in the background – it is important that internal and external colleagues understand why the guidelines are there and what they’re protecting us from
• at the end of the day it’s about people: what we do in publications will ultimately help HCPs make an informed choice and ensure patients get the right treatment.

Panel discussion

To finish session 1, Tim Koder (Oxford PharmaGenesis) moderated a panel discussion. Key points included:

• A cultural shift is needed, but this takes time, particularly in academia. The historical multimillion-pound lawsuits led to increased scrutiny within pharma – the issues with COVID-19 research could be the crisis needed to bring about further change.
• Guidelines, journals and publishing are all contributing to rebuilding trust – if one entity is lacking in integrity, trust in the whole system decreases.
• Defining predatory publishing is tricky because so many aspects need to be considered. Researchers should use their research skills to examine where they are publishing. The prevention of access to predatory publishing websites using technology is feasible but this is a big undertaking and touches on academic freedom – does a university have the right to stop an author accessing content?
• Preprints pose a challenge from a compliance point of view as demonstrated with COVID-19 research – results from preprints were picked up and reported by mainstream media without an appreciation of the preliminary nature of the research. This illustrates the need for public education around the limitations of science, research, and preprints.

——————————————————–

Written as part of a Media Partnership between EMWA and The Publication Plan, by Caroline Greenwood BSc and Beatrice Tyrrell DPhil from Aspire Scientific, an independent medical writing agency led by experienced editorial team members, and supported by MSc and/or PhD-educated writers.

——————————————————–

With thanks to our sponsor, Aspire Scientific Ltd

---