By David Tuller, DrPH
This month, the U.K. organization currently assessing whether to update the clinical guidelines for the illness it calls CFS/ME is seeking input from stakeholders in the process. Unfortunately, that’s not good news, it’s bad news.
Why? The National Institute for Health and Care Excellence, or NICE, is an independent body that operates under the sponsorship of the U.K. Department of Health and establishes guidelines for clinical practice. NICE created its current guidelines for CFS/ME, called CG53, in 2007. These were heavily influenced by the cognitive behavior therapy/graded exercise therapy (CBT/GET) paradigm, which was already dominant in Britain before the first PACE results were reported in The Lancet in 2011.
NICE is in the process of reviewing whether it should take a fresh look and update CG53. The guidelines cover a range of areas, including clinical presentation and diagnosis as well as management strategies. The organization has asked those who have registered as stakeholders, such as advocacy groups, to submit comments between July 10th and July 24th.
But in a recent exchange of e-mails with NICE, the ME Association clarified that the request for stakeholder input means the organization has already reached a preliminary decision that no revisions to the 2007 guidelines are necessary at this time. The two-week comment period therefore offers, at best, a formal opportunity to change the minds of people who have already indicated that their minds are made up. Since the available evidence has not yet persuaded them that recent events have rendered obsolete the old recommendations for CBT and GET, stakeholders have an enormous challenge in seeking to effect any change in their perspective.
The NICE solicitation of input from stakeholders comes right as the U.S. Centers for Disease Control has suddenly stopped recommending CBT and GET for what it calls ME/CFS. In early July, mention of the two treatments vanished from the CDC website. The website still suffers from multiple issues; the agency says more changes are forthcoming. But the removal itself should signal to NICE that the tide has shifted against both the CBT/GET ideology and the science purportedly proving that these treatments are effective. A decision to ratify the current NICE guidelines, as if nothing has changed since they were promulgated, will be greeted with dismay from many in the international scientific community.
What is obvious, but what NICE appears to overlook, is that the CBT/GET literature is plagued by two overarching flaws. The first is that these are mostly open-label trials with subjective outcomes, a study design rejected in other fields of medicine because it is so prone to bias. The second is that many of the studies, especially older ones, rely on the use of the Oxford criteria or other broad case definitions that yield heterogeneous samples because they conflate chronic fatigue and chronic fatigue syndrome. These studies simply cannot provide credible evidence about treatments for the specific disease that patients prefer to call ME. (PACE, of course, suffers from many other flaws besides these two.)
Some have argued that it is better to be able to provide patients with some recommendations rather than none. It is therefore important to stress that a recommendation should not be kept just because there are no better recommendations to make. If a recommendation is based on results from studies that promote bias, or on results that have been inflated through outcome-switching or have been derived from heterogeneous samples or are inaccurate for other reasons, then the recommendation needs to be rescinded, even if there is not sufficient current evidence to suggest other proven treatment approaches. That is especially the case when patient surveys and biomedical evidence raise concerns that a recommended strategy, in this instance, a steady increase in activity levels–is causing serious harm rather than benefit.
The fact that NICE decided to reconsider the 2007 guidelines this year seemed like a potentially promising development, but it was unclear what new information the agency would consider. For example, would the surveillance review include only reports from clinical trials of therapeutic interventions? Or would it also include findings of physiological abnormalities, most from research produced outside the U.K., which undermine the deconditioning theory that supports use of CBT and GET?
Other questions: Would the review consider the conclusion of the 2015 report from the U.S. Institute of Medicine (now the Academy of Medicine) that exertion intolerance was a cardinal symptom, which raises questions about whether GET is contra-indicated? Would it consider that the U.S. Agency for Healthcare Research and Quality found little or no evidence for CBT and GET after removing Oxford criteria studies from its analyses? And would it include the reanalyses of the reported PACE recovery and improvement findings, which were dramatically boosted by post-hoc outcome changes?
NICE’s surveillance proposal consultation document for CG53, recently posted on its website, provides answers to many of the questions. This 56-page report offers details about the NICE surveillance review and the reasons for the agency’s provisional decision not to change the guidelines. The review was apparently conducted by a NICE surveillance team with input from an unidentified number of unidentified topic experts. (I have filed a freedom-of-information request with NICE for the names of the experts it consulted. NICE has 20 working days to respond. The ME Association is also seeking the names, and the Countess of Mar has asked the Department of Health for the same information.)
The consultation document indicates that the surveillance team and topic experts did in fact take notice of the recent controversies and the new literature, including the reanalyses, before concluding that the guidelines should remain the same. The document noted that some upcoming research could impact the guidelines down the line, and identified specifically a study of online CBT for kids, an apparent reference to FITNET-NHS. This study exemplifies some of the problems common in this field of research, as I described on Virology Blog months ago. (Professor Esther Crawley of Bristol University, the trial’s lead investigator, subsequently referred to that blog post as libelous in a slide she showed during at least two speeches. She has not documented her charge.)
The consultation document also notes that only study abstracts, not the studies themselves, were reviewed. This is a surprising methodological choice given the significance of the issue. Abstracts can be seriously misleading and incomplete; studies themselves obviously provide a much more authoritative and nuanced picture. It does not seem too much to expect that those responsible for establishing enormously influential clinical guidelines should have taken the time to examine the actual research on which they were basing their recommendations. To learn that they did not is rather shocking.
In response to the controversy over the PACE trial, the document notes more than once that the investigators themselves have responded to criticisms, citing the FAQ on the trial website and other publications. The surveillance team appears to accept these responses at face value–as thorough and honest explanations. Perhaps no one has examined them closely enough to realize how empty and full of half-truths they are. The PACE investigators have certainly tried to defend their work. But there are no reasonable answers to many of the concerns, so their responses to date have only satisfied their ideological companions and those who know little about the debate.
The consultation document contains some troubling inaccuracies in its discussion of the PACE trial. For example, it reports that the PACE authors note that…changed thresholds for recovery were pre-specified. But it is simply false to call the revised thresholds “pre-specified.” The recovery thresholds for physical function and fatigue, two of the four recovery criteria in the 2013 paper published in Psychological Medicine–were the same as the normal range thresholds included in the 2011 Lancet paper. In that earlier paper, these thresholds were presented as part of post-hoc analyses, so it is hard to understand how the same thresholds could also be pre-specified under any standard definition of the term.
Besides that, all four recovery criteria were weakened, so it was self-evident that each change, whether pre-specified or not, would boost the numbers that the investigators could report had achieved recovery. Moreover, the 2013 paper does not cite any oversight committee approval for the major changes to the recovery definition–an oversight that should have raised alarm bells for Psychological Medicine. (Since the NICE surveillance team only reviewed abstracts and not actual papers, it would not have noticed this unusual lack of oversight committee approval.) And the consultation document fails to mention the fact that 13 percent of the PACE participants were already recovered for physical function at entry, an anomaly that should have prevented publication.
In another inaccurate (or at least highly disingenuous) statement, the consultation document notes that the investigators have re-analysed the main outcome measures according to the original protocol with similar results to those in the primary PACE results paper i.e. reduced fatigue and increased physical function. But it is stretching the truth beyond recognition to claim that the results of the reanalysis were similar. In the 2011 Lancet paper, the investigators reported that around 60 percent improved with CBT and GET; in the 2016 reanalysis, which used the original PACE protocol’s definition of improvement, the figure fell to around 20 percent.
Proponents of PACE have cited the 60 percent improvement rate as evidence of the effectiveness of CBT and GET. So a two-thirds decline in improvement rates should change any reasonable observer’s assessment of the effectiveness of the interventions. For NICE to accept the PACE investigators’ argument that this dramatic drop represents similar results–presumably because they were still able to report some very modest “improvement”–suggests that the surveillance team and topic experts are assessing the data with preformed opinions.
The topic experts seem to have enabled some of the NICE surveillance team’s own poor instincts. For example, after noting concerns raised about the Oxford criteria, the consultation document dismisses the significance of the issue this way:
Trials using Oxford criteria were eligible when developing NICE guideline CG53, and topic experts had no concerns about the inclusion criteria of trials in CFS. It was also noted by topic experts that there is no gold standard definition of chronic fatigue syndrome. There is currently insufficient consistent evidence about diagnostic methods for CFS/ME to determine an impact on the guideline recommendations.
Hm. So they used these studies the first time around ten years ago, and therefore it must be okay to use them again; something about that logic escapes me. And the topic experts expressed no concerns, shrugging off the case definition problem because there is no gold standard. But this thorny issue is at the core of the current controversy, and failure to address it is not a viable option. Scientists outside the influence of the CBT/GET ideological brigades understand very well that the populations generated with the Oxford criteria cannot yield actionable findings about an illness that should be defined much more specifically.
The consultation document also states that NICE will encourage Cochrane to update a 2008 review of CBT so that it can include the reported results from the PACE trial. A further review of the guideline may be considered following publication of the updated Cochrane review, stated the document. In other words, the NICE surveillance team is not only not deterred from considering the PACE results but is taking steps that would amplify their impact on the recommendations.
In fact, by citing Cochrane’s reviews as key evidence to support keeping the guidelines in place, the consultation document highlights the major role of these analyses in bolstering the entire CBT/GET enterprise. Cochrane takes the same problematic approach to assessing studies as NICE, accepting the results of open-label trials with subjective outcomes even though these are known to suffer from serious bias. In addition, Oxford criteria studies dominate the Cochrane reviews. In short, the body of research being used by both Cochrane and NICE, including but not limited to PACE, suffers from fundamental flaws. (Perhaps if Cochrane removed responsibility for the illness from the “common mental disorders group,” where it doesn’t belong, a new set of reviewers would demand higher-quality evidence.)
The provisional decision to not revise the guidelines at present has alarmed patients and advocates. The ME Association has launched an online petition, which notes that the intention is to offer the patient community the chance to demonstrate to NICE just how unhappy it is with the current guideline. The petition states:
As stakeholders, the ME Association believes that the current guideline is not fit for purpose and that we are not being afforded a fair opportunity to help produce a better version.
We believe the guideline should be improved to reflect international biomedical research and medical opinion and the overwhelming evidence against current treatment recommendations – specifically in relation to graded exercise therapy.
We also feel recent re-analysis of PACE Trial data and growing international concern about the efficacy of this research, and those therapies it examined, should result in a reappraisal of their use in this guideline.
The petition’s specific demands include, among others, a complete and proper review of the current guidelines, recognition of CFS/ME as a neurological disease in accordance with UK health agencies, removal of the GET recommendation in particular, and the highlighting of pacing as a strategy widely used by patients. The ME Association petition has so far collected more than 11,600 signatures.
NICE will presumably consider the stakeholder input before making its final decision. I guess the topic experts will also be given another chance to weigh in. I hope there’s a real chance to change the outcome.
**********
The CDC (Sort of) Explains
I sent the CDC a few questions last week after the agency had removed CBT and GET from its web pages, among other changes. I wanted to know what had prompted the changes, whether the removal of CBT and GET constituted an acknowledgement that there was no reliable evidence to support recommending the therapies, and whether the agency would apologize for having gotten it wrong. I also asked if the CDC would actively disseminate information about the changes.
Here’s what I heard back:
Dear Dr. Tuller,
Thank you for your interest in CDC’s recently updated ME/CFS web pages.
In follow-up to the IOM report, CDC hosted a roundtable meeting with a variety of stakeholders including ME/CFS patients, advocates, clinicians with ME/CFS expertise, healthcare professional organizations, medical educators, researchers/foundations, and other government agencies. The goal of this face-to-face meeting was to provide an opportunity for individuals to share their thoughts about how the IOM report could best be communicated through the CDC ME/CFS website and our educational materials for a variety of audiences.
During the website revision process, our staff focused on conveying information in plain language that would be understandable to the general public, regardless of the viewer’s background knowledge of ME/CFS. Part of this process included removing jargon and medical terms that we know are not widely understood by the general public and could be confusing.
As you will see on the website’s treatment tab, we suggest that patients with ME/CFS be careful about how they incorporate exercise into their lives. Every person’s experience with ME/CFS is unique so we continue to recommend that patients and their loved ones work closely with their doctor to determine the best course of management for themselves. We continue to believe that exercise can be useful for some ME/CFS patients, and also are trying to emphasize that people need to be careful not to overdo it and push themselves so far that it harms their health. We also state that talking to a therapist to help find strategies to cope with their illness may be helpful. We know from our past discussions with patients and their loved ones that there has been confusion about what we recommend related to exercise and therapy. We hope that the updated website provides more clarity.
CDC staff are currently developing the web pages for healthcare professionals. We plan to use social media and established partner communications channels to ensure that the general public and healthcare providers are aware of the new site once the healthcare pages are completed. In addition, we are collaborating with the CFSAC education workgroup on ways to disseminate information about ME/CFS to medical professionals and the general public.
Thanks again for your continued interest in this important public health topic.
This statement from the CDC is disappointing. The agency apparently does not plan to make a public announcement at this time that it has dropped CBT and GET, even though these therapies remain widely prescribed in the U.S. The agency will also not acknowledge that there is little or no credible evidence to support the treatments, and that it was wrong to recommend them for so long. The most it will concede is that there has been confusion about what we recommend related to exercise and therapy.
This is nonsense. Patients and advocates did not complain about the CDC website because they didn’t understand what it was recommending. They complained because it specifically recommended GET and CBT, citing PACE. After removing references to PACE from the website, the agency then claimed that CBT and GET were generic management strategies, unrelated to PACE. This claim was untrue. What other illness is GET recommended for? In what other illness is CBT supposed to alleviate you of the misguided belief that you have the illness in the first place?
Until early July, that is where things stood. Now the CDC seems to want to pretend it was all a big misunderstanding. To be clear: Patients are not against activity or exercise per se. They recognize that people with the illness should move their bodies to the extent they can without over-exerting themselves; that is a basic concept behind the self-management strategy called pacing. It is the graded aspect that patients find alarming, the idea that activity levels need to be steadily increased to reverse the deconditioning that is wrongly hypothesized as causing the symptoms. Similarly, patients are not against psychotherapy as a tool for coping with the demands and stresses they confront. They are against the specific form of CBT that has been created to treat their illness.
So patients were never confused by what the CDC’s recommendations meant. They understood that the recommendations meant the CDC bought into the CBT/GET paradigm, despite the PACE trial’s glaring flaws. In maintaining that it made the changes to prevent confusion, the CDC is weaseling out of admitting the obvious–the country’s top public health officials got this one wrong for years. The PACE trial–the crown jewel of CBT/GET research–has been authoritatively discredited. But the CDC won’t say so.
CBT and GET have been the de facto standards of care for this illness in the U.S. A major reason is that the CDC endorsed the therapies. Now the agency has abruptly un-endorsed them. That’s a big, big deal, not a minor tweak. For the sake of public health awareness and understanding, CDC officials have an obligation to provide a direct, transparent, and credible account of why they made these major changes. The statement I received from them does not demonstrate those qualities.
Comments are closed.