By David Tuller, DrPH
Last February, The BMJ published a paper called “Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial,” from McGregor et al. The study purported to have proven that this multi-disciplinary intervention was “clinically effective” in reducing symptoms associated with Long Covid. Unfortunately, the claim was fraught with problems that rendered it bogus. (I have written about the trial several times, including here and here.)
First and foremost, the study was unblinded and relied solely on subjective, self-reported outcomes—a combination of traits that inevitably leads to unknown amounts of bias. In such instances, modestly positive results would be expected as an artifact of the study design and are essentially meaningless.
Beyond that, the study included some untenable flaws. Specifically:
*In prominent sections of the paper, including the conclusion of the abstract and a highlights box called “What this study adds,” the investigators presented their findings as if they could be extrapolated to all Long Covid patients. This was completely unwarranted because it omitted a highly salient point. The study participants had all been hospitalized for acute Covid-19, while the great majority of Long Covid patients have not been hospitalized. Given the major differences between these two populations, it is scientifically unjustified to automatically assume that findings in one group apply to those in the other.
*The reported benefit on the primary outcome fell below the recommended threshold for what is called the “minimal clinically important difference” (MCID) for that measure, as determined by those who developed it. If a trial’s results do not meet the MCID recommended by the creators of an outcome measure, it is hard to take seriously the investigators’ claim that the intervention is “clinically effective.”
Given these and other issues, multiple complaints ensued. Some people filed rapid responses. I organized a letter to the journal’s editor-in-chief, Kamran Abbasi, signed by a dozen other colleagues. For their part, the investigators rejected all the criticisms in their own rapid response, posted last April. However, apparently someone at the BMJ disagreed with the investigators’ decision-making, because by May the paper bore a correction, although it addressed only the issue of the expansive extrapolation of the findings to all Long Covid patients. As the correction noted, the phrase “at least three months after hospital discharge for covid-19” was added to the key sections from which it had been omitted.
The correction did not include an explanation for why or how the journal overruled the investigators’ position that no such correction was needed. It was nonetheless an acknowledgement that the investigators, whether intentionally or not, had conveyed inaccurate or untrue information to the public. Unfortunately, media outlets had already disseminated this widely right after the study’s initial publication. No media outlets seemed to cover the correction.
Unfortunately for The BMJ, that is not the end of its responsibilities here. The claims from the trial have figured prominently in at least two other BMJ publications. An invited editorial accompanied the initial trial report last February. Like the trial itself, the editorial misrepresented the findings by not mentioning the limitations imposed by the study population until the very end. No one who reads it would necessarily be aware that a correction to the trial has severely restricted the relevance of the findings to the larger Long Covid population.
Even more troubling, The BMJ in November published an article called “Interventions for the management of long covid (post-covid condition): living systematic review,” from Zeraatkar et al. The review relies solely on McGregor et al to claim “moderate certainty evidence” in favor of a program of physical and mental health rehabilitation for Long Covid patients. The investigators rejected the notion that the recommendation should be limited to patients who were hospitalized, even though the review was accepted for publication months after REGAIN was corrected to reinforce that specific point.
Did anyone notice or care about this discrepancy? Hard to tell. The policy at The BMJ is to post peer reviews. In this case, no peer reviews of this systematic review have yet appeared, with no explanation offered for the delay. It goes without staying that if a study is corrected and its findings dramatically limited to a much smaller population, other articles that relied on the initial error should also be fixed—whatever the authors of those other articles think. Even the biopsychosocial fanatics apparently in charge at BMJ should be able to understand this basic principle.
I organized letters to The BMJ editor on both counts. (They can be read here and here.) In both cases, I received responses from BMJ’s “research integrity” department assuring me that the questions I had raised were being reviewed. I have not yet received any information about resolutions.
********
Meanwhile, there are now eight rapid responses appended to Zeraatker et al. (Embarrassingly, one is from the REGAIN investigators themselves, questioning the review’s statistical analysis.) Several raise concerns about the review’s over-broad claims regarding the REGAIN findings, among other problematic issues. Of particular note are smart, well-argued responses from two patient advocates, Michiel Tack and Dominic Salisbury.
Perhaps at some point The BMJ and the team that produced this problematic “living systematic review” will decide to provide some answers.