Trial By Error: A Bogus Request for Corrections to Recent Post on a Long Covid Exercse Study

By David Tuller, DrPH

Zachary Grin is a physical therapist in New York City who specializes in functional neurological disorder. Over the years, we had what I considered a good-natured, generally respectful exchange of views. As a gay man, I felt empathy for him—he posted about having difficulties with his parents after he came out.

But I blocked him a while ago after he accused me of lying. What was I “lying” about? The PACE trial. When I pointed out that some participants were “recovered” at baseline on physical function and/or fatigue, he argued that no one was “recovered” at baseline because there were four criteria they had to meet to be considered “recovered.” This, of course, is exactly the distraction and Trumpian logic the PACE authors themselves have used to avoid acknowledging the mess they created.

No one has reasonably argued that participants in PACE were “fully recovered” at baseline on all metrics. I certainly haven’t. So accusing me of that, as Zachary has, is a straw person argument. The PACE authors have refused for more than a decade to explain why ANYONE was “recovered” on ANY metric at baseline—much less the two primary outcomes of physical function and fatigue.

I explained all this to Zachary, noting that each of the four criteria had its own specific “recovery” threshold. So while no one was “fully recovered” at baseline—in other words, no one had met all four recovery thresholds—it was indisputably the case that some participants were “recovered” for physical function, fatigue or both.. Zachary continued to accuse me of lying and of deliberately misleading patients. That was the last straw. Gay brotherhood only goes so far.

After blocking Zachary, my social media has been blissfully Grin-free. Recently, however, I posted a critique of yet another stupid study on exercise for Long Covid—“Post-Hospitalisation COVID-19 Rehabilitation (PHOSP-R): A randomised controlled trial of exercise-based rehabilitation,” in European Respiratory Journal. Zachary posted a comment on the blog requesting “corrections” for non-existent errors. I’ve included here what he wrote, and my response to his silly and inaccurate arguments.

**********

Zachary’s comments on Virology Blog

David Tuller’s critique of the PHOSP-R study has several issues that misrepresent its findings.

He suggests healthcare providers may misapply the results to all Long Covid patients, despite the study’s clear exclusion criteria. The authors never claimed their findings applied universally, and clinicians understand the need to apply research appropriately.

Tuller also falsely claims the study doesn’t mention dropout reasons. Figure 1 lists them, including other commitments, unrelated illnesses, and rehabilitation offers. His speculation that patients may have found the intervention ineffective is unnecessary. High dropout rates are common in rehabilitation trials, and the study accounted for them through intention-to-treat and per-protocol analyses

He dismisses the statistically significant improvements in the Incremental Shuttle Walking Test by comparing participants to healthy populations instead of considering pre-intervention levels. The study never claimed participants would return to full health. The study aimed to measure improvements in exercise capacity, and those were evident. A study like this would not be expected to lead to a full recovery as cardiopulmonary adaptations take time – full recovery would likely take several months.

He dismisses that the study found potential clinical improvements in several secondary physical outcomes that surpassed the MCIDs. Though not statistically significant compared to usual care, this could be clinically relevant for rehabilitation.

He focuses on the remote intervention’s 34m ISWT improvement, claiming it doesn’t meet the 35m MCID and therefore the authors cannot claim clinical relevance. However, he fails to acknowledge a 1-meter difference is within the standard error of measurement. The per-protocol analysis showed a 42m improvement, reinforcing the intervention’s clinical relevance when participants stuck with the intervention. He cherry-picks alternative MCID estimates to cast doubt, despite different MCIDs applying to different populations. The 35m threshold is commonly used for this test. Considering this, the author’s conclusion is not false or an effort to “fudge” the facts. Remote care participants who adhered to the intervention experienced clinically relevant improvements in exercise capacity.

The PHOSP-R study provides evidence that both face-to-face and remote exercise-based rehabilitation lead to statistically significant and clinically meaningful improvements in post-hospitalization COVID-19 patients, within a well-defined patient population. Tuller’s review is not fair or rigorous which has resulted in a distorted evaluation that does a disservice to his readers, who fund his work and deserve better.

David, please consider making corrections to your post.

**********

My response to Zachary

As usual, Zachary’s comments are easy to rebut. And given his robust defense of the PACE trial’s fraudulent methodology (see above), I’m not surprised by anything he writes.  

1) Yes, the study was clear that it focused on post-hospitalized patients and excluded those with PEM. No argument there. But the conclusion, the “big picture” summation that many providers will rely on, does not. Whatever Zachary argues to the contrary, that omission clearly implies that their findings apply to all Long Covid patients. The idea that all health care providers will just know that these findings should only be extrapolated to patients without PEM, when the paper itself makes no such point, is not realistic. It also overlooks the well-recognized need for conclusions to be precisely worded so as to avoid any such misunderstandings.

2) Zachary writes that “Tuller falsely claims the study doesn’t mention drop-out reasons.” Because there is a table with pro forma reasons for dropping out, Zachary says any suggestion that participants found the intervention “unnecessary.” High drop-out rates are common in intervention trials, he says, so what’s the big deal? Plus, they compensated for the drop-out rates with intention-to-treat analyses.

To start with, Zachary’s charge itself is false. I didn’t write that the study didn’t mention reasons for dropping out. I wrote that there is “no substantive discussion of the drop-out rate.” Furthermore, the reasons for drop-out listed in the table tell me nothing about whether participants found it unhelpful or a waste or even harmful. I think it’s fair to assume that participants are more likely to complete an intervention if they found it helpful and less likely if they did not—whatever other reasons for dropping out they might have mentioned.

The authors could have argued, as Zachary does, that these drop-out rates are normal and nothing to fuss about. They didn’t. Conducting an intention-to-treat analysis helps adjust the findings to a point but tells us nothing about those who didn’t finish.

3) Whatever the study claimed or didn’t claim about its goals, the notion that patients remained almost as severely disabled after the intervention as before, when compared to results on the primary outcome for healthy people of similar ages, is nowhere mentioned in the study. Just because the authors didn’t declare that the intervention would leave to full recovery doesn’t mean they have no obligation to place their findings in a realistic context.

This problematic omission creates a misleading impression about what the reported results mean in practical terms for patients. It helps explain one of the paper’s anomalies—the null results in health-related quality of life scores. These suggest that the claimed “effectiveness” of the intervention was so minimal that patients did not experience the impact as an actual benefit. These null results in health-related quality of life are apparently meaningless to Zachary.

4) This study included many, many findings, but the authors did not make any corrections for for multiple tests. Given that, and the high drop-out rate, statistically insignificant secondary findings are of more interest to Zachary than to me.

5) Let’s be clear: 34 meters, the primary outcome for the remote intervention arm in the intention-to-treat analysis, is lower than 35, the MCID cited for the ISWT. And the intention-to-treat analysis, not the per-protocol analysis, is the relevant one here. But Zachary thinks it’s fine to claim a result met the MCID in such circumstances when the MCID is within the standard error, or when a per-protocol analysis surpasses the MCID. He is wrong.

That’s not to say these aren’t factors to be considered. The authors are free to explain that the intention-to-treat primary outcome fell marginally short of the MCID and then point out what Zachary has. They’re not free to claim their results met the MCID for the remote intervention when the key outcome did not, even if the difference was only a meter.