Trial By Error: Some Good News on Cochrane

By David Tuller, DrPH

In what can only be characterized as a welcome surprise, Cochrane has rejected the revision of a 2014 review of exercise treatments for chronic fatigue syndrome, stating that the work does not meet the organization’s quality standards. Cochrane revealed the decision late Friday in a statement appended to the review, which itself was a revision of a review first published years earlier.

That review reported that exercise therapy, most often graded exercise therapy, was effective. Yet the review was a methodological mess. Two patients–Tom Kindlon and the late Robert Courtney, wrote extensive and cogent critiques after it was published. When the lead author of the review provided inadequate responses, Courtney filed a formal complaint with Cochrane.

Cochrane found merit in the complaint and had recently given the authors a chance to revise their work accordingly. This was the revision that Cochrane has now deemed unsatisfactory.

Why did Cochrane’s announcement surprise me? I had assumed the revision would be unsatisfactory. But Cochrane had initially decided to withdraw the review pending these revisions, and had backed down from doing so under what must have been enormous pressure from the CBT/GET ideological brigades and their defenders. So the review has not been withdrawn during this revision process.

Given that early retreat on Cochrane’s part, I was concerned the editorial team would require the authors to make only a few tweaks but no substantive changes to the review. Luckily, I was wrong. Here’s the statement Cochrane posted on Friday:

The author team has re-submitted a revised version of this review following the complaint by Robert Courtney. The Editor in Chief and colleagues recognise that the author team has sought to address the criticisms made by Mr Courtney but judge that further work is needed to ensure that the review meets the quality standards required, and as a result have not approved publication of the re-submission. The review is also substantially out of date and in need of updating. 

Cochrane recognises the importance of this review and is committed to providing a high quality review that reflects the best current evidence to inform decisions. 

The Editor in Chief is currently holding discussions with colleagues and the author team to determine a series of steps that will lead to a full update of this review. These discussions will be concluded as soon as possible.

The suggestion that the review is out of date is interesting. This could be referring to the fact that raw PACE trial data were released two years ago, and there are now published reanalyses of the findings. Perhaps Cochrane recognizes that these reanalyses, which refute the positive results reported by the PACE team, would need to be included in any credible full update.

The Cochrane statement does leave some questions unanswered. Given that a revision was deemed necessary in the first place, the current published version is obviously unsatisfactory as is. Will this published but unsatisfactory version be withdrawn? If not, why not? If so, when?

Moreover, the Cochrane statement seems conveniently ambiguous on a key point. Will the authors of the current review be asked to develop the full update themselves or will they just be participating in discussions of what steps should be taken going forward? Until given reason to think otherwise, I will interpret that ambiguity as deliberate and therefore as a positive sign.

Perhaps this is the moment when all, including the authors, will agree that the time has come to remove this illness from the Common Mental Disorders group. In conjunction with that, perhaps all will agree as well to withdraw the reviews conducted under the auspices of this group, the exercise review as well as an older review of cognitive behavior therapy for the illness.

In fact, the response to a comment posted on the review in October suggests that transferring the illness elsewhere could be a real possibility. A reader raised once again the concern about the illness being housed within the Common Mental Disorders group, citing the 2015 report from the Institute of Medicine and the WHO’s categorization of it as neurological. Cochrane’s response was not from the lead author of the review but from two representatives of the Common Mental Disorders group itself. Here’s part of what they wrote:

In response to concerns raised by members of the CFS/ME community, Cochrane has been considering repositioning the editorial oversight of CFS/ME reviews. The Cochrane CMD Review Group currently sits within the Brain, Nerves and Mind (BNM) Network. In the future, reviews on this topic might sit with another Cochrane Review Group within the BNM Network, or they might transfer to another Network altogether…Please be reassured that this is currently under consideration and a decision is anticipated before the end of 2018.

Given the source, perhaps this response is a signal that the Common Mental Health Disorders group has agreed to a collegial and imminent hand-off of responsibilities and that an acceptable biomedical home will be found for ME/CFS. That would indeed be an excellent development.

Comments are closed.

Scroll to Top