By David Tuller, DrPH
Same-Day Update: In re-reading the new paper, I noticed that the discussion section also features errors involving the percentages. It includes this sentence: “About 9% of individuals who were not working at baseline had returned to work at follow-up.” And this one: “Further, 6% of those working at baseline were no longer working at follow-up.” Both of these statements are categorically wrong. These are percentages of the total sample, not of the sub-groups of those working and not working at baseline.
**********
The discredited PACE trial of psychological and behavioral interventions for chronic fatigue syndrome (CFS) included four objective outcomes, a six-minute walking test, a step-test for fitness, employment/educational status, and whether the person was receiving social benefit. All four outcomes failed to match the positive reports on the subjective measures of fatigue and physical function. That’s why the authors completely ignored them in assessing the efficacy of the interventions and declared victory anyway.
But they went further than that, after the fact, they actually appeared to dismiss the objectivity and relevance of their own objective measures. This little-noticed effort to explain away the bad results occurred in the authors’ response to correspondence related to their 2013 paper on €œrecovery from the illness, which was published in Psychological Medicine.
Here’s what they wrote in response to criticism that their assessment of “recovery” should have taken into account the null results for changes in employment status:
€œRecovery from illness is a health status, not an economic one, and plenty of working people are unwell€¦while well people do not necessarily work. Some of our participants were either past the age of retirement or were not in paid employment when they fell ill. In addition, follow-up at 6 months after the end of therapy may be too short a period to affect either benefits or employment. We therefore disagree€¦that such outcomes constitute a useful component of recovery in the PACE trial.
Ok, then. Never mind those pesky, unattractive results for our objective outcomes!
Now Professor Chalder and colleagues have published a new paper in the journal Occupational Medicine called €œChronic Fatigue Syndrome and Occupational Status: A Retrospective Longitudinal Study. And here’s what they write: €œStudies into CFS have placed little emphasis on occupational outcomes, including return to work after illness.
The authors call for more attention to this domain but do not mention that PACE included such an outcome and had null results. This is a disturbing and indefensible omission, given that the PACE trial was hailed as the “definitive” investigation of their favored interventions.* [This sentence has been corrected. I originally wrote that the PACE authors themselves called it the “definitive” investigation. The use of the word I remembered was actually from a statement about the trial disseminated by the Science Media Centre.] Unfortunately, given the authorship of this new paper, this serious lapse and the hypocrisy involved are unsurprising.
**********
Mangling the data
In the new study, the researchers looked at 316 attendees of a CFS clinic who completed information on their employment status both at baseline, that is, before treatment, and at follow-up. The average follow-up period was around nine months.
The study found that 53% of the participants were working at both baseline and follow-up, while 33% of the participants were not working at either baseline or follow-up. In addition, 6% of the total sample were working at baseline but not at follow-up, while 9 % were not working at baseline but were at follow-up.
But both the abstract and the section on €œkey learning points mangle these data by introducing confusion about the denominators of the percentages. In the abstract, the authors write the following: €œPatients were followed up for an average of 285 days and over this period 53% of patients who were working remained in employment. Of the patients who were not working at baseline, 9% had returned to work at follow-up. However, of those working at baseline, 6% were unable to continue to work at follow-up.
Similarly, here are some statements from the section called €œkey learning points: €œOver half of the patients who were working at baseline were able to remain in work over the follow-up period and 9% returned to work. However, of those working at baseline, 6% were unable to continue to work at follow-up.
All of these statements self-evidently misstate the statistics. After all, if only 53% of those working at baseline were still working at follow-up, the obvious corollary is that 47% of those working at baseline were no longer working at follow-up. But a simple glance at the table of results reveals the error, the authors don’t seem to understand that the 53% figure refers to a proportion of the entire sample of 316, not just to a proportion of those working at baseline.
Moreover, the 9% of participants who had “returned to work” after not working at baseline were 9% of the total sample, not 9% of “the patients who were not working at baseline.” And the 6% of participants who were working at baseline but not at follow-up was 6% of the total sample, not 6% “of. those working at baseline.”
It is perplexing that experienced investigators would have so much trouble accurately explaining their own findings. It is also perplexing that no peer reviewers or journal editors caught these substantive mistakes. Clearly, these statements need to be formally corrected. Also, the journal clearly needs to investigate why its peer review process failed so dramatically.
Moreover, the article states that patients were assessed using the clinical guidance from the National Institute for Health and Care Excellence (NICE). (The authors are presumably referring to the document issued in 2007, not the revision published in October of this year.) But the reference for this statement is not to NICE’s 2007 guidance but to a 1991 paper outlining the so-called €œOxford criteria for CFS, which required only six months of unexplained fatigue for a diagnosis.
So which is it? Did the clinic use the NICE criteria, or the Oxford criteria? And if it used the NICE criteria, why was the Oxford criteria paper referenced? This, too, needs to be corrected.
Interestingly, the authors do reference the PACE study in connection to the phrase €œrandomized controlled trials in this sentence: €œThe 9% of patients returning to work in this study is heartening and suggests that people can recover, as previously found in both randomized controlled trials and routine clinical practice. In other words, the authors are once more claiming that the interventions in PACE led to €œrecovery.
As usual, the authors promote their perspective with zero acknowledgement that serious challenges have knocked their theoretical approach and treatment paradigm off its once-vaunted pedestal. They do not point out that the PACE claims of €œrecovery have been rebutted in published reanalyses and rejected by the new NICE guidelines. They also seem, again, to have forgotten that Professor Chalder and her PACE colleagues have already dismissed the argument that employment status should be regarded as an aspect of €œrecovery from the illness.
So Professor Chalder is contradicting herself and talking in circles here. She has once more demonstrated her incompetence and her deficient understanding of science. Sad.
Comments are closed.