By David Tuller, DrPH
*This is a crowdfunding month at UC Berkeley. If you’d like to support my work, the link is here.
**********
When it comes to exercise and Long COVID, investigators have routinely claimed success for their interventions–and many news articles have followed their lead in covering the issue. Most reporters don’t have the time or bandwidth to look more deeply and challenge that narrative—in particular, by checking out whether the much-touted trials really live up to their hype.
A story this week in New Scientist, a London-based publication, breaks this mold. Journalist Alexandra Thompson did her homework. She had approached me a few months ago, indicating that she was working on a piece about how exercise was emerging as a possible treatment for Long COVID. I pointed out that it might appear from the claims of investigators and accompanying media reports that exercise was effective, but that the science did not really support this conclusion.
We spoke at length, and I knew she was talking with other critics of this body of research as well as with its proponents. In circumstances like these, you can only hope a story turns out reasonably well.
So I was pleased when I saw the story today with the following headline: “Exercise advice for long covid may be doing more harm than good.” And this subhead: “Exercise has been touted as a tool for managing and treating long covid, but much of the evidence has neglected one of its most debilitating symptoms: post-exertional malaise.” In the story, Thompson quotes, among others, Irish patient advocate and researcher Tom Kindlon; Todd Davenport, a professor of physical therapy at the University of the Pacific in Stockton, California; and me.
Here’s the opening:
“In the hunt for ways to alleviate long covid—a relatively new condition with no cure, experienced by millions of people worldwide after contracting covid-19—exercise has been a bright spot. It’s drug-free, it costs nothing and a handful of studies have suggested it boosts long covid recovery. But concern is growing that these studies aren’t robust enough to support exercise as a treatment approach, reigniting a decade-long controversy over the use of exercise in addressing other conditions, such as chronic fatigue syndrome.”
The story then highlights a Scottish trial from the University of Glasgow, identifying it as “one of the highest-profile studies of exercise in long covid.” The trial, which received positive media attention when it was published last November by JAMA Network Open, purported to prove that a program of “resistance exercise therapy” was helpful. An article disseminated on Yahoo, headlined “Long-COVID Patients Should Focus on This Training,” included the following quote from the study’s lead investigator: “Our study shows the benefits of strength training for recovery after COVID-19 and suggests that people suffering from persistent symptoms after a COVID-19 infection could benefit from this type of training.”
The primary outcome was a standard measure of exercise capacity called the Incremental Shuttle Walk Test (ISWT), in which subjects start off walking slowly and gradually pick up speed. As evidence of the intervention’s purported effectiveness, the investigators noted that those in the intervention arm improved an average of 83 meters on the ISWT from baseline, compared to an increase of 47 meters for those in the comparison arm.
But the trial protocol had designated 46 meters as the threshold for what was considered a clinically significant benefit, and the statistically adjusted difference of 36.5 meters between the two groups was significantly lower than that. In other words, the results were statistically significant but clinically insignificant. In the real world, the difference between the outcomes in the intervention and comparison arms was too small to make any meaningful difference to patients, or even to be noticeable. per the trial’s own assessment methods.
As I wrote in a blog post at the time:
“Here’s the main problem for the investigators and their primary outcome. Per the statistical analysis plan outlined in their protocol, the ISWT’s threshold for clinical significance—known as the minimal clinically important difference (MCID)—is 46 meters. The investigators cited an authoritative 2022 paper from the European Respiratory Society (ERS)–“Use of exercise testing in the evaluation of interventional efficacy: an official ERS statement”–for this MCID. They used it to informed their power calculations for determining the needed sample size…
“In the trial, participants in the intervention arm performed better than the control group by an average of 36.5 meters. Let’s acknowledge the obvious: 36.5 meters is quite a bit lower than the MCID of 46 meters that the investigators themselves referenced in their statistical analysis plan. Oops!
“Researchers with a robust sense of integrity would have reported such a salient detail, however disappointing or embarrassing or contrary to their expectations. Yet the JAMA Network Open paper made no mention of it, effectively disappearing a very revealing data point. This omission, in and of itself, serves to misrepresent the findings and is arguably a form of research misconduct. Readers deserve to know that the trial’s primary outcome did not reach its own threshold for clinical significance.”
Most readers of the study, or readers of articles about the study, would not know these important facts. To Thompson’s credit, she has drawn attention to them, in the process raising questions about the ubiquitous promotion of exercise for Long COVID. As she writes in the New Scientist article:
“The study quickly garnered press coverage and was widely discussed on social media. But many scientists have pointed out issues with this trial…The difference between the distances walked by the control and exercising groups fell 10 metres short of the minimum threshold for clinical significance chosen by the team at the start of the experiment. ‘If you don’t achieve the level that’s minimally clinically important, you don’t go around claiming success,’ says David Tuller at the University of California, Berkley [sic]. In response, Berry [Colin Berry, the lead investigator] says it’s not for us to say whether an individual would benefit from this improved mobility. ‘I think that’s open to interpretation.’”
Colin Berry’s response doesn’t make a lot of sense. What, exactly, is “open to interpretation”? The facts are the facts–the exercise intervention did not offer participants, on average, clinically significant benefits on the primary outcome, according to the trial’s predesignated metrics. Yet Berry and his colleagues chose not to mention this key detail about the threshold for clinical significance in their published report. The decision to withhold this information was ethically and scientifically indefensible.
The New Scientist article also discusses the failure of Long COVID exercise trials to adequately address post-exertional malaise. It addresses the overlaps between ME/CFS and Long COVID and provides some back-history of the PACE trial debacle. It’s great to see New Scientist taking a deeper look at some of the usual claims.
So, did short covid “emerge” prior to regular-length covid, and thereafter long covid emerged … or is that just another stretch?
That IS good news. Thanks for your input, David.
the ridiculous thing is if you walk with me and have me train sure I’ll do more, I’ll be motivated to.
you’ll just not be around later the next day when I feel my muscles die off.
thankfully I found some stuff that limits that effect but God beware I go over the 10 or 15 min exercise mark. even just walking for an hour has been highly involved with later crashes. the last left me immobile for a week. one e scooter ride over bumpy roads and two walks with friends was all it took. I still got inflamed and itching joints.
exercise? my ass. a daily 10min walk is mandatory but to gauge symptom strength and energy levels for the day. it’s also the top of the mountain as far as performance goes.
exercise and overexertion is the way to degrade.
exercise in rehab contexts gives minimal but pretty on paper results because it’s the only activity in the day and many hours of recovery follow it. and if you overdo it? they told me 1 hour of gym was recommended for LC cases. so I did 1hr on the bike and damn I loved to move. 4 days crash followed. I asked them how that was supposed to work out. and then the doctor told me, oh no, we only meant you’re supposed to find your limits. dear doc, I knew those since long because I live with this daily. I found them long ago when I broke down because all my skeletal muscles stopped working. I told you. but my stay at rehab is filed as a success because when they asked leading questions mid conversation if I felt it was good for me that I was there, I didn’t say it was 100% useless, since I had had 5 weeks to catch my breath and mostly just be sick. did I improve? no, they had nothing in their therapy program for that anyway. is it counted as a success for LC rehab? oh sure.
I’m glad this kind of reporting is reaching a broader audience, because too many long COVID exercise stories get framed as hopeful before they are careful. For a lot of patients the issue is not fear of movement, it is that post exertional worsening, orthostatic symptoms, and brain fog can make the wrong kind of exertion actively destabilizing. Better scrutiny helps because it pushes clinicians and researchers to stop treating a very mixed patient population like one simple rehab problem.
I tried several different physical therapists who refused to listen to me about how most cardio exercise would make me crash and be so sore and achy, and sometimes even just minimal strength training exercises, especially when combined with heat. I told them that only massage seems to help but they just kept dismissing me and continued to push exercise. And honestly I can do exercises on my own when I know I can handle them.
One of the first times it was blatantly obvious was a couple of summers ago, in the late afternoon in NYC, one physical therapist had me do 10 min on a stationary bike and I felt super faint and dizzy before I even finished 7 mins. I then threw up at the physical therapy office, and felt so weak and sick for the next few hours. Yet, next time, they kept pushing strength training no matter how much I told them about my PEM crashes and how bad I felt after I left their office when I threw up.
After a few more of these similar experiences even though I insisted on not doing the same thing, I just stopped going to physical therapy altogether.
Last week, I did 8 minutes of a 10 min very easy pilates video I had been doing for the past 10-12 years with ease and even in my first couple of years of long covid with no noticeable issues but last week, I had a horrible crash from it. Yet some days, I can go to a dance party for a few hours and have some achiness the next day but nothing as bad as the crashes from the pilates or the physical therapy workouts. It’s mostly unpredictable but I have come to realize that if I’m already not feeling well, then I can’t push myself. I have learned to listen to my body but it’s amazing how all the physical therapists refused to listen to me. I really appreciate this article as it is evidence against the fact that exercise is the answer to such a complex and unpredictable chronic illness!