By David Tuller, DrPH
On Thursday, Professors Vincent Racaniello and Mady Hornig, both from Columbia University, wrote to BMJ’s research integrity coordinator. I have been corresponding with BMJ, and specifically the research integrity coordinator, about the Norwegian study of cognitive behavior therapy combined with music therapy as a treatment for chronic fatigue in adolescents after acute EBV infection (known in the US as mononucleosis and elsewhere as glandular fever). BMJ Paediatrics Open published the paper a few months ago.
Besides many issues with the paper itself (see below), the peer review process seems to have broken down. BMJ Paediatrics Open has open peer reviews, and the reviews are posted on the journal’s website. In this case, one of the two peer reviewers wrote in his review that he did not read “beyond the abstract.” (He provided some notes on the abstract.)
For reasons that haven’t been explained, the reviewer’s admission that he didn’t review the actual paper was not an obstacle to publication for BMJ Paediatrics Open. It is not clear if editors read the peer review and decided it didn’t matter that the reviewer didn’t read the paper, if they read the review and didn’t notice the relevant statement, or if they didn’t read the review at all. Whatever the explanation, the lapse represents a major peer review failure for BMJ.
Along with colleagues from Columbia, Berkeley and University College London, I sent a letter to BMJ alerting editors this problem as well as multiple methodological and ethical concerns involving the paper itself. That was almost two months ago. Yet BMJ Paediatrics Open has still not warned readers that this paper did not pass BMJ’s own strict standards for peer review. Nor has it offered a deadline for resolving the self-evident problems with the paper. What is BMJ waiting for?
This casual approach to addressing critical matters of scientific integrity seems to belie the point of having a research integrity coordinator. It also suggests a perplexing indifference to the health of children suffering from a serious post-viral condition–at a time when the planet is engulfed in a viral pandemic with potentially major long-term consequences.
Besides the broken peer review process, the study suffered from multiple flaws, including but not limited to the following:
1). The trial protocol, registration and statistical analysis plan all described the research as a fully powered trial. But recruitment proved difficult, the intervention group experienced high attrition, and the results were disappointing. The published paper presented the research as a feasibility study without mentioning that it was, in fact, designed as a fully powered trial. The investigators suggested in their conclusions that the results “might justify a full-scale clinical trial”, even though they had just conducted such a trial and it had failed to generate the findings they wanted.
2). “Post-exertional malaise” was highlighted as an outcome in the published paper but was not mentioned in the trial registration, protocol or statistical analysis plan.
3). The primary outcome was average steps per day, objectively measured. Both groups performed worse on this measure after the trial than they did at baseline, with the decline even greater among those who received the intervention. In arguing in their conclusion that the findings “might justify a full-scale clinical trial,” the investigators omitted reference to these poor results for their pre-designated primary outcome.
4) The investigators constructed a definition of “recovery” that ignored the objectively measured primary outcome and relied solely on a subjective secondary outcome. Then they presented these questionable “recovery” data only in a per-protocol analysis rather than an intention-to-treat analysis. The per-protocol analysis by definition failed to account for the high attrition rate in the intervention group, leading to an inflated reported “recovery” rate.
In their letter to the BMJ’s research integrity coordinator, Professors Racaniello and Hornig note that continued delay in this matter is not acceptable. As they write, “It seems clear that BMJ has an obligation to inform readers immediately that the paper did not pass a proper peer review, and, assuming the methodological lapses documented by Dr. Tuller are confirmed, to retract the paper.”
The letter was cc’d to the other co-signers of the initial letter (Professor Jonathan Edwards of University College London, Professor John Swartzberg of UC Berkeley, and me). Also cc’d were Professor Imti Choonara, BMJ Paediatrics Open‘s editor-in-chief; Dr Fiona Godlee, BMJ’s editorial director; and Ingrid Spilde, a Norwegian journalist who has written about the study.
**********
Dear Ms. Ragavooloo–
We hope you are managing well through these complicated times.
With increasing reports of persistent post-COVID-19 symptoms and concerns about the potential for these post-viral syndromes to develop into ME (concerns acknowledged by NIAID Director, Dr. Anthony Fauci, 9 July 2020, https://edition.cnn.com/world/live-news/coronavirus-pandemic-07-09-20-intl/h_5125152a01f8c98d362cf15d6860ab37) physicians are increasingly looking to authoritative sources to learn about post-viral fatigue as well as ME and how to best manage them.
Ensuring that the publications to which readers will turn for clinical guidance have passed proper peer review and be free of study should certainly be a matter of vital importance at the BMJ.
It is in this context that we wish to underscore the importance of rapid resolution of the matters to which Dr. Tuller, along with us and other colleagues, first called your attention on 31 May 2020.
It seems clear that BMJ has an obligation to inform readers immediately that the paper did not pass a proper peer review, and, assuming the methodological lapses documented by Dr. Tuller are confirmed, to retract the paper.
We thank you in advance for your considered attention and prompt action.
Sincerely,
Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, New York, USA
Mady Hornig, MA, MD
Associate Professor of Epidemiology
Columbia University Mailman School of Public Health
New York, New York, USA
**********
By the way, if you noticed a glitch in the third paragraph of the letter, good for you! I am told that the phrase “free of study” should have read “free of protocol deviations, ethical anomalies and other issues.” Oops! I assume BMJ got the point in any event.
Comments are closed.