By David Tuller, DrPH
Three years ago this month, Virology Blog published my 15,000-word investigation of the PACE trial, so this seems like a good time for a bit of reflection. I certainly didn’t expect this saga to drag on this long. I’ve kept at it mainly because of the UK academic and medical establishment’s unwillingness to acknowledge what’s gone wrong, its failure to provide reasonable responses to substantive criticism, and its persistence in defending indefensible studies.
I’ve often been asked why I started this quixotic effort to examine the research behind the interventions that have served as the standard of care for the illness or cluster of illnesses variously known as ME, CFS, ME/CFS and CFS/ME. (I realize the questions of nomenclature are divisive and that people hold many different views on the matter.) After all, I have no family members with this devastating condition. My only connection to it before I got involved was through a long-time friend who was diagnosed in the early 1990s.
So how did I latch onto this issue? In 2009, I became intrigued by the controversy over whether a retrovirus might be associated with the illness and started reporting about it. In early 2011, with the retrovirus debate still unresolved, The Lancet published the first results of the PACE trial, which reported that cognitive behavior therapy and graded exercise therapy were effective treatments. That was the first time I’d heard of the study. Two years later, Psychological Medicine published a follow-up paper on purported recovery from these two therapies.
At the time, I was mainly focused on other endeavors. But eventually I read commentaries about PACE written by patients and advocates and published as blog posts, letters to The Lancet and other journals, and reports on various sites. The more I read, the more compelling and convincing I found their criticisms of the trial and its methodology. In contrast, the responses from the PACE authors seemed incoherent and non-responsive. Since I am not an epidemiologist or a biostatistician, I consulted with colleagues at Berkeley, Columbia and elsewhere before tackling the issue. These experts were universally shocked at the poor quality of the research.
It is not really difficult to figure out that the PACE trial is an unacceptable piece of science. Anyone with a basic awareness of human nature should be able to grasp that unblinded trials relying on self-reported outcomes, like PACE, are so fraught with bias as to be uninterpretable. That’s why clinical trials with this design are never used as the basis for making decisions about pharmaceutical therapies, and why the strategy of blinding researchers and participants to treatment allocation was developed in the first place.
Of course, it is difficult if not impossible to blind behavioral and psychological treatments, such as the therapies tested in PACE; that’s why it is essential to combine them with objective outcomes. In PACE and other trials in this body of ME/CFS research, objective outcomes have largely failed to support the subjective reports of success. Yet those null findings have been dismissed or ignored by the crowd of avid CBT/GET proponents.
Moreover, trials in which participants have met the thresholds for key outcome variables before they even begin treatment have no place in the published literature. That a study including this unusual feature and other major failings has achieved wide acceptance is, in the words of Carol Monaghan, a member of Parliament from Glasgow, one of the biggest medical scandals of the 21st century. That’s why more than 100 experts signed the most recent open letter to The Lancet. The letter, sent and posted on Virology Blog in August, cited the PACE trial’s unacceptable methodological lapses and requested a fully independent investigation.
Some patients report that CBT, GET, and a range of other treatments, including the Lightning Process, have helped them improve or even recover from ME/CFS. But in the absence of high-quality evidence, such anecdotal claims should not be the basis for public health policy or clinical guidelines. And the PACE study and related trials cannot be construed as high-quality evidence, no matter how many times Professor Sir Simon Wessely, Professor Michael Sharpe, the UK Medical Research Council, the Science Media Centre and other influential British individuals and organizations repeat such claims.
**********
My work has obviously not pleased those I have criticized. But the pooh-bahs of UK science and medicine have likely been offended not only by my interpretation of the research but my manner of expressing myself. I have called PACE a piece of crap, and sometimes worse. I have suggested that the trial seems to meet the Medical Research Council criteria for research misconduct. I have sent sharply worded letters to respected editors at major journals about their inadequate oversight processes and their ongoing refusal to acknowledge indisputable errors and scientific missteps.
I have referred to PACE supporters as members of the CBT/GET ideological brigades. At two public events, I tore a print-out of the PACE trial into pieces and tossed them in the air. I have suggested that the unwarranted adherence to the PACE/CBT/GET approach is a form of insanity. I have said that the situation is a real-life case of the emperor-has-no-clothes and that supporters of this treatment paradigm have been stark naked for years.
My tone and behavior might be atypical in scholarly debate, and I have obviously violated British conventions of decorum and deference. But I don’t care much about politeness and niceties, especially given the stakes for patients. In any event, the domain of social commentary provides much greater latitude for robust argumentation. More importantly, the accuracy of my reporting has withstood the scrutiny to which it has been subjected by those who dislike my views. I have made minor mistakes here and there, as everyone does. Unlike the PACE authors, I have corrected errors when others have pointed them out or when I have noticed them myself. This is standard practice in journalism but apparently not in UK science.
**********
The most recent front in this international scientific dispute revolves around Cochrane, the organization known for conducting systematic reviews of interventions for a wide range of diseases and conditions. Earlier this month, it was reported that Cochrane had decided to temporarily withdraw its 2014 review of exercise treatments, which found them to be effective. This decision was based on solid scientific and methodological grounds, but I assume that CBT/GET supporters must have been pulling out their hair in distress at the news. They had undoubtedly been exerting enormous pressure and lobbying hard behind the scenes to prevent something like this from happening.
In contrast, patients and advocates were jubilant. They had been expressing concerns about this review since it was published. In particular, two very smart patients, Tom Kindlon and the late Robert Courtney, submitted extensive critiques that were appended to the review when it was republished last year. They pointed out that the authors violated Cochrane’s clear guidelines on how to rate studies for risk of bias, failed to include objective outcomes that would have undermined the review’s conclusions, and changed the assessment strategies described in their own protocol in ways that allowed them to present more positive results. When the lead author provided unsatisfactory responses, Courtney filed a formal complaint with Cochrane.
Many people besides patients have been troubled by this review’s obvious problems. Dozens of scientists, clinicians and other experts signed Virology Blog’s recent statement of support for Cochrane’s decision. When I met with Cochrane editor-in-chief David Tovey this summer, I vigorously pressed the arguments advanced by Courtney, Kindlon and others. From the tenor of this interaction, it was apparent that some sort of action to address the review’s flaws would likely be forthcoming, so I wasn’t surprised at the news.
Yet the media accounts ignored the detailed scientific reasoning that formed the basis of the decision. Instead, they focused on patients’ purported militancy and supposed bullying. The charge that Cochrane made the choice because of patient pressure rather than valid criticism is way off-base. But it is no more surprising than the fact that the powers-that-be have for years hailed this anti-scientific research as authoritative.
Cochrane’s decision was an enormous victory for proper science and a possibly fatal blow to the fortunes of this approach to treatment. That’s why it is disappointing that the intense pushback from the CBT/GET ideological brigades has apparently forced a retreat. Instead of temporarily withdrawing the review, Cochrane has added a notice indicating that the authors will submit a revised version for assessment, most likely later this year. In the meantime, the review stands. Whether the upcoming revision will resolve the serious questions raised by Courtney, Kindlon and others remains to be seen.
.
***********
Finally, a few thoughts about my crowdfunding. Last year, a grant that had supported my position at Berkeley ended, as scheduled. Anyone who has worked at an American public university knows that many if not most positions are funded through outside sources of money. The University of California system has been in financial crisis for years, given the state’s abysmal levels of support. So I decided to crowdfund, as is increasingly common in academia. I have conducted two successful crowdfunding campaigns. As a result, I have been criticized in some quarters for having questionable conflicts of interest and for being paid specifically to trash the PACE trial.
In fact, I am being paid to continue my investigation of research into behavioral and psychological interventions for this terrible illness. I’m not responsible for the fact that PACE and so many other studies deserve to be tossed into the dustbin. I pursued this Virology Blog project for two years as an unpaid public service. That work involved three trips to England and one to the Netherlands, all self-funded. During that time, I received $6000 for an 8,000-word piece in Undark, the MIT-based online science magazine, and $100 or so (I can’t remember the exact amount) for my half of the fee for a New York Times opinion piece I co-authored.
No journalist or public health researcher would have embarked on this journey to make money or enhance his or her professional status. I decided to do what I’ve done because those outside the ME/CFS world were ignoring these serious problems and leaving debilitated patients to fend for themselves. I also recognized that the CBT/GET recommendations were based on terrible science and were potentially causing harm, since patients can suffer serious relapses after trivial amounts of exertion. I crowdfunded to support a half-time position, yet I am spending most of my work life on the effort. To suggest that I have taken on this issue and am criticizing PACE for financial gain is ridiculous.
Comments are closed.