By David Tuller, DrPH
Last month, I sent a letter to Dr Abbasi, the editor-in-chief of The BMJ, about an egregious and arguably fraudulent claim in the REGAIN trial—an investigation of a Long Covid physical and mental health rehabilitation program. Several other experts co-signed the letter.
The journal asked us to send in a rapid response. We declined, on the grounds that the journal had already ignored a rapid response making a similar point, which had been posted weeks before. Moreover, we were not interested in debating the issue. The facts were the facts. We considered it the journal’s and the authors’ responsibility to correct the paper.
Of course, nothing has happened since. So earlier today I sent the letter again, this time with two dozen signatories.
I should make clear that I am not at all surprised, given my previous experience with Dr Abbasi and his flawed decision-making. Before his current position, he was the editor of the Journal of the Royal Society of Medicine. During his tenure, the journal published a piece of crap whose authors included Professor Trudie Chalder and Professor Sir Simon Wessely, a one-time president of the royal society. In that case, the paper made causal claims about clinical outcomes while disavowing the possibility of making causal claims from the available data set. As with the REGAIN trial report, the authors contradicted themselves.
Brian Hughes, a professor of psychology at the Univesity of Galway, and I wrote to the journal seeking a correction. Dr Abbasi rejected our request. We ended up publishing a critique of the paper in the Journal of Health Psychology. Everyone can make stupid mistakes. Failing to correct them constitutes research misconduct—or, in this case, the editorial variety. To stand behind clear misstatements, as Dr Abbasi did in that instance and is doing now with REGAIN, means the papers themselves can only be called deceitful and fraudulent.
That this deceitfulness and fraudulence is occurring at the highest levels of UK academic medicine is shocking—or perhaps it is more accurate to say it should be. In this domain, very little shocks me anymore. That doesn’t make the behavior any less creepy. But at this point in time, no one should be surprised by anything Trump does. (Oops–I mean Dr Abbasi and other sycophantic supporters of the CBT-and-exercise ideological brigades. Apologies to former president Trump!!)
**********
Dear Dr Abbasi:
In February, The BMJ published “Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial,” by McGregor et al [1]. In the trial, 585 patients experiencing prolonged symptoms after having been hospitalized for covid-19 were assigned to either an eight-week intervention, consisting of more than ten one-hour online sessions, or to “usual care,” consisting of one half-hour online consultation.
The trial was unblinded and relied solely on subjective outcomes—a study design guaranteed to generate an unknown amount of bias. The primary outcome, health-related quality of life at three months, was assessed with the preference score (PROPr) derived from the Patient-Reported Outcomes Measurement Information System (PROMIS) [2]. While no biomarkers are available for “post-covid-19 condition,” the authors could have chosen to include among their secondary outcomes one or more objective measures of function, such as the six-minute walking test, step-test for fitness, actigraphy to assess distance moved, or employment status.
Given the bias inherent in the study design, it is surprising that the findings for the primary outcome were nonetheless so poor that the authors apparently felt compelled to misrepresent them. In the main intention-to-treat analysis, the authors report a mean difference in PROPr scores between the intervention and usual-care arms of 0.03 at both three months and 12 months. (The latter was a secondary outcome.) In the abstract’s conclusion, the authors state that the intervention was “clinically effective at improving health related quality of life at three and 12 months compared with usual care” in “adults with post-covid-19 condition.”
To begin with, it is inappropriate for the authors to extrapolate findings from patients who were hospitalized with covid-19 to the much larger number of patients with prolonged symptoms who were not hospitalized. Beyond that serious error of interpretation, the reported difference of 0.03 in PROPr scores between the two groups does not support the assertion that the intervention was “clinically effective.”
In the description of PROPr in the methods section, the authors note that “as with other preference based measures such as the EuroQol 5 dimension 5 level (EQ-5D-5L) instrument, a difference of 0.03 to 0.05 is considered to be clinically important.” However, the reference for this statement, an official “frequently-asked-questions” (FAQ) page for the PROMIS measures being used in the study, explicitly contradicts the authors’ claim that “a difference of 0.03 to 0.05 is considered to be clinically important” for PROPr [3].
Here is the explanation from the FAQ cited by the authors:
“The minimally important difference for PROPr has not been formally evaluated, but we currently recommend using 0.04. Most preference-based measures have minimally important difference thresholds between 0.03 and 0.05. Current work-in-progress suggests a minimally important difference of 0.04 would be appropriate, although a conservative estimate of 0.08 (half of a standard deviation in the PROPr dataset) could also be used.”
In other words, the authors’ assertion in the methods section that the accepted threshold for a “clinically important” or “minimally important” difference for PROPr is the same as for other preference-based measures is categorically false [4, 5]. In the article’s discussion section, the authors acknowledge as much but then provide an irrelevant rationale for having claimed clinical effectiveness anyway.
As they write: “Research completed since we started this study suggests a minimally important difference of 0.04 on the PROPr score between groups. Our observed differences of 0.03 (95% confidence interval 0.01 to 0.05) at three months and 0.03 (0.01 to 0.06) at 12 months are smaller than this suggestion. However, the complier average causal effect analysis showed a larger effect of 0.05 (0.01 to 0.09) at three months and 0.06 (0.01 to 0.10) at 12 months, suggesting that the true effect, in those fully complying with the intervention, might exceed this threshold.”
This argument is unsustainable. The authors cannot legitimately claim clinical effectiveness for their overall findings if this claim only applies to a more fully compliant subset of their sample and is not borne out by the main analysis. The paper—and in particular the abstract’s conclusion that the intervention was “clinically effective” for health-related quality of life—must be corrected to reflect that the primary outcome results did not meet the currently recommended threshold for “clinically important” or “minimally important” difference for the measure in question. (I have cc’d the co-signatories and the lead trial investigator.)
Sincerely,
David Tuller (corresponding author)
Center for Global Public Health
University of California, Berkeley
Berkeley, California, USA
davetuller@berkeley.edu
Nicola Clague-Baker
School of Health Sciences
University of Liverpool
Liverpool, England, UK
Svetlana Blitshteyn
Dysautonomia Clinic
Buffalo, New York, USA
Todd Davenport
Department of Physical Therapy
University of the Pacific
Stockton, California, USA
David Davies-Payne
Department of Radiology
Starship Children’s Hospital
Auckland, New Zealand
Jonathan Edwards
Department of Medicine
University College London
London, England, UK
Andrew Ewing
Department of Chemistry and Molecular Biology
University of Gothenburg
Gothenburg, Sweden
Mark Faghy
Human Sciences Research Centre
University of Derby
Derby, England, UK
Keith Geraghty
Centre for Primary Care and Health Services Research
University of Manchester
Manchester, England, UK
Cori Hayden
Department of Anthropology
University of California, Berkeley, USA
Calliope Hollingue
Department of Mental Health
Johns Hopkins Bloomberg School of Public Health
Baltimore, Maryland, USA
Mady Hornig
Department of Epidemiology
Columbia University Mailman School of Public Health
New York, New York, USA
Brian Hughes
School of Psychology
University of Galway
Galway, Ireland
Leonard Jason
Center for Community Research
DePaul University
Chicago, Illinois, USA
David Joffe
Respiratory and Sleep Medicine
Royal North Shore Hospital
Sydney, Australia
Binita Kane
Respiratory Medicine
Manchester University NHS Foundation Trust
Manchester, England, UK
Douglas Kell
Institute of Systems, Molecular and Integrative Biology
University of Liverpool
Liverpool, England, UK
Asad Khan
North West Lung Centre
Manchester University Hospitals
Manchester, England, UK
Resia Pretorius
Department of Physiological Sciences
Stellenbosch University
Stellenbosch, South Africa
David Putrino
Department of Rehabilitation and Human Performance
Icahn School of Medicine at Mt Sinai
New York, New York, USA
Charles Shepherd
ME Association
Gawcutt, England, UK
Michael Stingl
Neurology Department
Votivpark Specialist Medical Center
Vienna, Austria
John Swartzberg
Division of Infectious Diseases and Vaccinology
School of Public Health
University of California, Berkeley
Berkeley, California, USA
Susan Taylor-Brown
Department of Pediatrics, Developmental & Behavioral Pediatrics
University of Rochester Medical Center
Rochester, New York, USA
1. McGregor G, Sandhu H, Bruce J, et al. Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial. BMJ 2024;384;e076506
2. Dewitt B, Jalal H, Hanmer J. Computing PROPr Utility Scores for PROMIS® Profile Instruments. Value Health 2020;23:370-8
3. PROPr: The PROMIS-Preference Score. What is this minimally important difference for PROPr? https://www.proprscore.com/faqs/; 2023
4. ME/CFS Skeptic. Thread: Clinical effectiveness of an online group physical & mental health rehab programme for post-covid-19 condition REGAIN study, 2024, McGregor+. Science for ME [Internet]; 9 Feb 2024; #23. Accessed at: https://www.s4me.info/threads/clinical-effectiveness-of-an-online-group-physical-mental-health-rehab-programme-for-post-covid-19-condition-regain-study-2024-mcgregor.37174/page-2#post-514588
5. Louise S. Rapid Response: Re: Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial — Conclusions are concerning. BMJ; 13 Feb 2024. Accessed at: https://www.bmj.com/content/384/bmj-2023-076506/rr
Writing from the path of the eclipse, in tears with a broken heart. The last eclipse, seven years ago, was a magical time. My son’s last summer at home, or so we thought, before heading to UChicago. While I watched the eclipse from home, my son was help little day campers safely view. Now, he’s been home five years in bed with ME. How dare these researchers play with our lives. They are not true scientists. They are unscrupulous thieves of the little research dollars that are out there and thieves of hope.
I wonder – do medical scientists and journal editors these days ever stop to consider the harm they could cause if they give out false or misleading information? I’d say that standards in science are crucial but in medical science should be paramount. Misinformation can cost lives so medical researchers need to abide by the rules of science and report accurately and not hype their findings, while journal editors must ensure that standards are rigorously adhered to by all involved in creating and reviewing articles. Medical science should not be regarded as an interesting sideline or some game to be played. It’s deadly serious. Dodgy doctors might endanger tens or hundreds or thousands of patients in their ‘care’ but murky medical science could risk the health of millions. (And the greater the journal’s impact factor, the greater the risk, I imagine.)
It looks to me like medicine has a crisis on its hands. Many thanks to all who signed the letter.