By David Tuller, DrPH
On Friday, I had an e-mail exchange with Sir Andrew Dillon, chief executive of the NICE Guidance Executive. The other seven Guidance Executive members are various directors within the NICE hierarchy, including the communications director. This group will make the final decision about whether to accept the provisional decision of a NICE surveillance review team to leave as is CG53, the guidance for CFS/ME released in 2007. (I have written about the NICE review process on CG53 here, here and here.)
That ten-year-old CFS/ME guidance recommends treatment with graded exercise therapy and cognitive behavior therapy. NICE reaffirmed the guidance after the 2011 publication of the first PACE results, which were taken as evidence that these treatments were effective. As part of the current review process, NICE provided stakeholders with a two-week window last month to submit comments about the provisional decision not to change CG53. Not surprisingly, this recommendation has alarmed many patients and advocates.
I didn’t expect to get answers to my questions from the Guidance Executive, but I felt an obligation to pose them anyway, given the importance of the issues. In fact, Sir Andrew responded to my e-mail within an hour. He explained that no comments would be forthcoming while the Guidance Executive was reviewing the situation. I have posted his response below my initial e-mail.
**********
Sir Andrew Dillon
Chief Executive
National Institute for Health and Care Excellence
Dear Sir Andrew:
I am a journalist and public health researcher at the University of California, Berkeley. I have reported on the current review of CG53, the NICE guidance for CFS/ME, for the science site Virology Blog, which is hosted by Professor Vincent Racaniello, a microbiologist at Columbia University. I have previously reported for Virology Blog on the PACE trial and other issues related to graded exercise therapy and cognitive behavior therapy. Earlier this year, I co-authored a commentary about the serious problems with PACE for the Sunday opinion section of The New York Times.
In my role as a journalist covering this issue, I have some questions for you and the other members of the NICE Guidance Executive about the decision-making process concerning the provisional recommendation to make no changes to CG53:
1) For many years, the U.S. Centers for Disease Control recommended GET and CBT as treatments, citing PACE. In late June or early July, the agency removed all references to these therapies from its main pages on the illness. Does the Guidance Executive plan to consult with American public health officials about what prompted this major dis-endorsement of these two therapies that NICE continues to promote?
2) In 2015, both the U.S. National Institutes of Health and the Institute of Medicine (now the National Academy of Medicine) released reports on the illness (they call it ME/CFS). These reports both concluded that it is a serious organic disease involving pathophysiological processes and not a psychological or psychiatric disorder, a determination that would have significant impact on treatment options. Does the Guidance Executive plan to consider these two reports and consult with any of the members of the panels that wrote them?
3) Other fields of medicine have abandoned the use of the trial design favored in this entire body of research, including PACE: open-label studies with subjective outcomes. That’s because other fields of medicine recognize that the combination of those two features in one study inherently produces bias. Does the Guidance Executive share these concerns about results from open-label studies with subjective outcomes, or does it believe that such studies can produce reliable and unbiased evidence suitable for clinical decision-making?
4) In PACE and other studies from this field, objective measures have largely failed to support the subjective results that have generated claims of recovery or significant clinical improvement. Does this pattern of sharp contradiction between objective and subjective results raise questions for the Guidance Executive about whether patients are objectively getting better?
5) In the 2011 Lancet paper, 13 % of the PACE participants had already met one of the study’s outcome thresholds at trial entry, that is, although assessed as disabled enough in physical function to qualify for the study, they were also found to be within normal range for physical function, before any treatment at all. In the 2013 Psychological Medicine paper, the same 13 % were already recovered for physical function at baseline, before any treatment at all, that is, they were simultaneously disabled for physical function and recovered for physical function. These facts were not included in the published papers but emerged later through a patient’s freedom-of-information request. Does the Guidance Executive have confidence in the reported results of a study in which a significant minority of participants have already met a key outcome threshold at baseline? If so, can the Guidance Executive point to other studies in the clinical trial literature in which a significant number of participants have already met a key outcome threshold at baseline? Does the Guidance Executive believe that the published PACE papers should have mentioned the fact that a signifiant minority of participants had already met a key outcome threshold at baseline?
6) In February 2016, forty-two leading scientists and clinicians signed an open letter to The Lancet in which they outlined the methodological lapses of the PACE trial, stated unequivocally that such flaws have no place in published research, and demanded an independent investigation. In March 2017, more than 100 experts signed an open letter to Psychological Medicine, asking the journal to retract immediately its core finding that GET and CBT helped patients recover. Does the Guidance Executive plan to review these open letters and consult with any of the signatories–from Columbia, University College London, Harvard, Stanford, Berkeley, etc., about their reasons for publicly dismissing the PACE findings as invalid?
7) Both GET and CBT, as described in PACE and other studies from this field of research, involve telling participants that the treatments can reverse the illness and return them to a state of health. Is the Guidance Executive concerned that telling study participants repeatedly about the effectiveness of the treatments could bias their responses, augmenting any bias already inherent in open-label studies with subjective outcomes?
8) Some defenders of PACE note that CBT is also recommended for patients with cancer and other chronic diseases. But the approach advocated in PACE and related studies is not the kind of CBT focused on helping patients adapt to the reality of their illness. Rather, this form of CBT is specifically designed to alleviate patients of their purportedly unhelpful beliefs of having an ongoing medical disease that can be exacerbated by activity and exercise. Is the Guidance Executive aware of this critical distinction between CBT as normally administered in the case of other chronic illnesses and the adapted form of CBT investigated in PACE and other studies in this field?
9) The PACE trial used the Oxford criteria to identify participants. This case definition requires only six months of unexplained fatigue, so its use could result in the selection of participants with depression or other unidentified fatiguing illnesses. Some of these other illnesses might resolve spontaneously or respond to behavioral and psychological interventions like GET and CBT. In fact, the NIH report noted that using the broad Oxford case definition could impair progress and cause harm, and recommended that it be retired. Is the Guidance Executive concerned that populations derived using the Oxford criteria might contain many participants experiencing prolonged fatigue for a range of reasons unrelated to the illness being investigated? Is the Guidance Executive concerned that such heterogeneity in study samples could lead to erroneous findings about treatments?
10) The U.S. Agency for Healthcare Research and Quality found evidence to support GET and CBT for ME/CFS in its review of multiple studies. However, when the agency subsequently removed Oxford criteria studies from this analysis, it found no evidence that GET provided any benefits and almost no evidence that CBT provided benefits. Is the Guidance Executive considering this AHRQ re-analysis in its decision-making? Does the Guidance Executive plan to consult with officials at the agency to discuss why they conducted this re-analysis and how it subsequently led them to downgrade their assessments of the therapies?
11) The surveillance review team cites Cochrane reviews of GET and CBT to support the recommendation to leave the 2007 guidance as is. Many of the trials included in these Cochrane reviews rely on a broad case definition like the Oxford criteria. Is the Guidance Executive comfortable relying on Cochrane reviews for confirmation of controversial findings when the reviews themselves include the studies that feature the methodological problems being questioned? Will the Guidance Executive consider asking Cochrane to follow the lead of American public health officials and conduct a re-analysis of its GET and CBT reviews with Oxford criteria studies removed from the sample?
12) In the PACE trial protocol, the investigators promised to follow the Declaration of Helsinki, which requires researchers to tell prospective participants about any possible conflicts of interest. The three main PACE investigators have had longstanding relationships with insurance companies, advising them to offer GET and CBT to claimants diagnosed with the illness. Yet the investigators did not tell prospective PACE participants about these extensive consulting and financial links with insurance companies or include the information in consent forms. Is the Guidance Executive concerned that this clear violation of the investigators’ protocol promise to disclose “any possible conflicts of interest to prospective participants means that they did not obtain properly informed consent? Does the Guidance Executive believe it should base clinical guidelines on studies that have not obtained properly informed consent?
13) More than 15,000 people signed the ME Association’s online petition outlining their concerns with the 2007 guidance and their objection to the provisional decision to leave it unchanged. Is it unusual for that many people to sign a petition protesting a NICE guidance?
14) Surveys of patients who have undergone GET have routinely found that more patients report harms from the intervention than benefits. In making its decision, does the Guidance Executive plan to consider these reports based on the clinical experiences of patients receiving GET in the real world?
15) The conduct and findings of the PACE trial have become a worldwide controversy. The study has been presented as a paragon of bad science at conferences of epidemiologists and statisticians and in graduate-level seminars. Leading scientists and clinicians have publicly denounced the trial’s perplexing irregularities. The CDC has removed references to PACE and has dropped the associated treatment recommendations. In making its decision about the 2007 guidance, does the Guidance Executive plan to consider that large segments of the scientific and public health worlds have already rejected the evidence base for GET and CBT as interventions for CFS/ME, ME/CFS, or whatever the disease entity is called? Given the public health stakes involved, will the Guidance Executive consider commissioning a more extensive, authoritative, independent and unbiased review of the evidence–and perhaps even a review in which the reviewers read the actual studies on which they are basing their recommendations, and not just the study abstracts?
I have other questions, but will leave it at that for now. I would be delighted should you and/or other members of the Guidance Executive choose to respond.
Kind regards–David
David Tuller, DrPH
Senior fellow in public health and journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
**********
Sir Andrew’s réponse to me:
Dear Dr Tuller,
Thank you for your enquiry.
It looks like you are aware that we have recently concluded a public consultation about our provisional decision on the review of this clinical guideline. We are in the process of reviewing the results of that consultation and will make our final decision in due course. We will make that decision public, together with any other statements we think will be helpful to contextualise it. Until then, we don’t intend to respond to enquiries about the provisional decision. It may be that our final decision, when placed in the public domain, will help you with some of your questions, but if not, we will endeavour to answer them as best we can at that time.
Yours sincerely,
Andrew Dillon
Chief Executive
National Institute for Health and Care Excellence
10 Spring Gardens | London | SW1A 2BU | United Kingdom
Comments are closed.