Trial By Error: The New Interferon “CFS” Study

By David Tuller, DrPH

I haven’t had time to cover the new and wildly over-hyped study about prolonged fatigue–and purportedly about “chronic fatigue syndrome”–that was published this week in the journal Psychoneuroendocrinology. Thanks no doubt to the involvement of the Science Media Centre, this mildly interesting piece of research has received widespread media attention.

Since the study team included confirmed members of the biopsychosocial ideological brigades, I remain wary of the motives of those involved. Even more so since this group of researchers apparently continues to define the illness using the discredited Oxford criteria, as they do in the Psychoneuroendocrinology study. This case definition requires only six months of unexplained fatigue and is known to generate heterogeneous populations of people suffering from a range of fatiguing disorders.

Given these findings, will biopsychosocial researchers soon be promoting GET and CBT as appropriate therapies for the reported immune dysfunction? Or are they so worried that the field is moving beyond their discredited “deconditioning/unhelpful beliefs” model that they’re desperate to stake a claim on some biomedical aspect?

This paper documents that a particular form of immune activation can lead to a prolonged state of fatigue, even after the stimulus has ended. It is unwarranted for the authors, the SMC or anyone else to make anything other than extremely speculative and cautious claims at this point about any relevance of these findings to ME/CFS. Instead, in presenting the results, they have dramatically overstated the case–and, as usual, the media herd has followed their lead.

Here’s a handy overview of the study from MEAction:

https://www.meaction.net/2018/12/18/post-hepatitis-fatigue/

And here’s an insightful post from Nick Brown, a psychology graduate student at the University of Groningen in the Netherlands. Brown was profiled earlier this year in Science magazine as one of the “data thugs” taking down poor-quality research through reanalyzing results.

I’m posting Nick’s blog in full below, with his permission.

**********

Have scientists found an explanation for the onset of ME/CFS?

In this post I’m going to discuss this article (the link leads to a page from which you can download the PDF; I’m not sure if this will last), which appeared today (17 December 2018):

Russell, A., Hepgula, N., Nikkheslat, N., Borsini, A., Zajkowska, Z., Moll, N., . . . Pariante, C. M. (2018). Persistent fatigue induced by interferon-alpha: A novel, inflammation-based, proxy model of chronic fatigue syndrome. Psychoneuroendocrinology. Advance online publication. http://dx.doi.org/10.1016/j.psyneuen.2018.11.032

(Notes for nerds: (1) The article date will become 2019 when it appears in print; (2) There are 20 named authors, so I’m glad that APA referencing style only requires me to list the first six and the last one. I will be calling it “the article” or “the study” or “Russell et al.” henceforth.)

Before I start, a small disclosure. In 2015, a colleague and I had a manuscript desk-rejected by Psychoneuroendocrinology for what we considered inadequate reasons. This led to a complaint to the Committee on Publication Ethics and a change in the journal’s editorial policies, but unfortunately did not result in our article being sent out for review; it was subsequently published elsewhere. My interest in the Russell et al. article arose for entirely unrelated reasons, and I only discovered the identity of the journal after deciding to look at it. So, to the extent that one’s reasoning can ever be free of motivation, I don’t believe that my criticisms of the article that follow here are related to the journal in which it appeared. But it seems like a good idea to mention this, in case the editor-in-chief of the journal is reading this post and recognises my name.

Media coverage

This article is getting a fair amount of coverage in the UK media today, for example at the BBC, the Mail Online, the Independent, and the Guardian (plus some others that are behind a paywall). The simplified story that these outlets are telling is that “chronic fatigue syndrome is real and is caused by [the] immune system” (Mail Online) and that the study “challenges stigma that chronic fatigue is ‘all in the mind'” (Independent). Those are hopeful-sounding message for ME/CFS patients, but I’m not sure that these conclusions are justified.

I was made aware of this article by a journalist friend, who had received an invitation to attend a press briefing for the article at the Science Media Centre in London on Friday 14 December. By a complete coincidence I was in London that morning and decided to go along. I was allowed in without a press pass after identifying myself as a researcher, but when I tried to get clarification of a point that had been made during the presentation I was told that only reporters (i.e., not researchers or other members of the public) were allowed to ask questions. This was a little annoying at the time, but on reflection it seems fair enough since time is limited and the event was organised for journalists, not for curious researchers with a little time on their hands. There were about 10 journalists present, from most of the major UK outlets.

You can get a summary of the study from the media pieces linked above (the Guardian’s coverage by Nicola Davis is particularly good). If you haven’t seen the media articles, go and read them now, and then come back to this post. There was also a press release. I suggest that you also read the Russell et al. article itself, although it does get pretty technical.

What did the study claim to show?

Here’s my summary of the study: The participants were 55 people with hepatitis C who were about to undergo interferon-alpha (IFN-α) treatment. The treatment lasted 24, 36, or 48 weeks. At five time points (at the start of treatment, than after 4 weeks, 8 weeks, 12 weeks, and at the end of treatment, whenever that was), patients were asked about their levels of fatigue and also had their cytokine levels (a measure of activity in the immune system) tested. These tests were then repeated six months after the end of treatment. Patients were also assessed for depression, stressful life events, and childhood trauma.

Interferon-alpha occurs naturally in the body as part of the immune system, but it can also be injected to fight diseases in doses that are much greater than what your body can produce. It’s sometimes used as an adjunct to chemotherapy for cancer. IFN-α treatment often has substantial fatigue as a side effect, although this fatigue typically resolves itself gradually after treatment ends. But six months after they finished their treatment, 18 of the 55 patients in this study had higher levels of fatigue than when they started treatment. These patients are referred to as the PF (“persistent fatigue”) group, compared to the 37 whose fatigue more or less went away, who are the RF (“resolved fatigue”) group.

The authors’ logic appears to run like this:

1. Some people still have a lot of fatigue six months after the end of a 24/36/48-week long course of treatment with IFN-α for hepatitis C.

2. Maybe we can identify what it is about those people (one-third of the total) that makes them slower to recover from their fatigue than the others.

3. ME/CFS patients are people who have fatigue long after the illness that typically preceded the onset of their condition. (It seems to be widely accepted by all sides in the ME/CFS debate that a great many cases occur following a infection of some kind.)

4. Perhaps what is causing the onset of fatigue after their infectious episode in ME/CFS patients is the same thing causing the onset of fatigue after IFN-α treatment in the hepatitis C patients.

Russell et al.’s claim is that patients who went on to have persistent fatigue (versus resolved fatigue) at a point six months after the end of their treatment, had also had greater fatigue and cytokine levels when they were four weeks into their treatment (i.e., between 46 and 70 weeks before their persistent fatigue was measured, depending on how long the treatment lasted). On this account, something that happens at an early stage of the procedure determined how well or badly people would recover from the fatigue that was induced by the treatment once the treatment was over.

Just to be clear, here are some things that Russell et al. are not claiming. I mention these partly to show the limited scope of their article (which is not necessarily a negative point; all scientific studies have a perimeter), but also to make things clearer in case a quick read of the media coverage has led to confusion in anyone’s mind.

– Russell et al. are not claiming to have identified the cause of ME/CFS.

– Russell et al. are not claiming to have identified anything that might cure ME/CFS.

– Russell et al. are not claiming to have demonstrated any relation between hepatitis C and ME/CFS.

– Russell et al. are not claiming to have demonstrated any relation between interferon-alpha, whether this is injected during medical treatment or naturally produced in the body by the immune system, and ME/CFS. They do not suggest that any particular level of IFN-α predicts, causes, cures, or is any other way associated with ME/CFS.

– Russell et al. are not claiming to have demonstrated any relation between a person’s current cytokine levels and their levels of persistent fatigue subsequent to interferon-alpha treatment for hepatitis C. (As they note on p. 7 near the bottom of the left-hand column, “we … find that cytokines levels do not distinguish [persistent fatigue] from [resolved fatigue] patients at the 6-month followup”.)

– Russell et al. are not claiming to have demonstrated any relation between a person’s current cytokine levels and their ME/CFS status. (As Table 2 shows, cytokine levels are comparable between ME/CFS patients and the healthy general population.)

Some apparent issues

Here are some of the issues I see with this article in terms of its ability to tell us anything about ME/CFS.

1. This was not a study of ME/CFS patients

It cannot be emphasised enough that none of the patients in this study had a diagnosis of ME/CFS, either at the start or the end of the study, and this greatly limits the generalisability of the results. (To be fair, the authors go into some aspects of this issue in their Discussion section on the left-hand side of p. 8, but the limitations of any scientific article rarely make it into the media coverage.) We don’t know how long ago these hepatitis C patients were treated with interferon-alpha and subsequently tested at follow-up, or if any of them still had fatigue another six or 12 months later, or if they ever went on to receive a diagnosis of ME/CFS. One of the criteria for such a diagnosis is that unresolved fatigue lasts longer than six months (it would have been really useful to have had a further follow-up). But in any case, the fatigue that Russell et al. studied was, by definition, not of sufficient duration to count as “chronic fatigue syndrome” (and, of course, there are several other criteria that need to be met for a diagnosis of ME/CFS; chronic fatigue by itself is a lot more common than full-on ME/CFS). I feel that it is therefore rather questionable to refer to “the presence of the CFS phenotype… for … IFN-α-induced persistent fatigue” (last sentence on p. 5). Maybe this is just an oversight, but even the description of persistent fatigue as “the CFS-like phenotype”, used at several other points in the article, is also potentially somewhat loaded.

Furthermore, the patients in this study were people whom we would have expected to be fatigued, at least throughout their treatment. IFN-α treatment knocks you about quite a bit. Additionally, fatigue is also a common symptom of hepatitis C infection, which makes me wonder whether some of the patients with “persistent fatigue” maybe just had a slightly higher degree of fatigue from their underlying condition rather than the IFN-α treatment — the definition of persistent fatigue was any score on the Chalder Fatigue Scale that was higher than baseline, presumably even by one point (and, theoretically, even if the score was 0 at baseline and 1 six months after treatment ended). So Russell et al. are comparing people who are recovering faster or slower from fatigue that is entirely expected both from the condition that they have and the treatment that they underwent, with ME/CFS patients in whom the onset of fatigue is arguably the thing that needs to be explained.

There are many possible causes of fatigue, and I don’t think that the authors have given us any good reason to believe that the fatigue that was reported by their hepatitis C patients six months after finishing an exhausting medical procedure that itself lasted for half a year or more was caused by the same mechanism (whatever that might be) as the multi-year ongoing fatigue in ME/CFS patients, especially since, for all we know, some or all of the 18 cases of persistent fatigue might have been only marginal (i.e., a small amount worse than baseline) or resolved themselves within not too many months.

2. Is post-treatment fatigue really unrelated to cytokine levels?

It can be seen from Table 2 of the article that the people with “persistent fatigue” (i.e., the hepatitis C patients who were still fatigued six months after finishing treatment) still had elevated cytokine levels at that point, compared to samples of both healthy people and ME/CFS patients. Indeed, these cytokine levels were similarly high in patients whose fatigue had not persisted. The authors ascribe these higher levels of cytokines to the IFN-α treatment; their argument then becomes that, since both the “resolved fatigue” and “persistent fatigue” groups had similar cytokine levels, albeit much higher than in healthy people, that can’t be what was causing the difference in fatigue in this case. But I’m not sure they have done enough to exclude the possibility of those high cytokine levels interacting with something else in the PF group. (I must apologise to my psychologist friends here for invoking the idea of a hidden moderator.) Their argument appears to be based on the assumption that ME/CFS-type fatigue and post-IFN-α-treatment fatigue have a common cause, which remains unexplained; however, in the absence of any evidence of what that mechanism might be, this assumption seems to be based mainly on speculation.

3. Statistical limitations

The claim that the difference in fatigue at six-month follow-up was related to a difference in cytokine levels four weeks into the treatment does not appear to be statistically robust. The headline claim — that fatigue was greater after four weeks in patients who went on to have persistent fatigue — has a p value of .046, and throughout the article, many of the other focal p values are either just below .05, or even slightly higher, with values in the latter category being described as, for example, “a statistical trend towards higher fatigue”, p. 4. But in the presence of true effects, we would expect a preponderance of much smaller p values.

Russell et al. also sometimes seem to take a creative approach to what counts as a meaningful result. For example, at the end of section 3.1, the authors consider a p value of .09 from a test to represent “trend-statistical significance” (p. 4) and at the start of section 3.2 they invoke another p value of .094 as showing that “IL-6 values in [persistent fatigue] subjects … remained higher at [the end of treatment]” (p. 5), but in the sentence immediately preceding the latter example, they treat a p value of .12 as indicating that there was “no significant interaction” (p. 5).

These borderline p values should also be considered in the light of the many other analyses that the authors could have performed. For example, they apparently had all the necessary data to perform the comparisons after eight weeks of treatment, after 12 weeks of treatment, and at the end of treatment, as well as the four-week results that they mainly reported. None of the eight-week or 12-week results appear in the article, and the two from the end of treatment are extremely unconvincingly argued (see previous paragraph). It is possible that the authors simply did not perform any tests on these results, but I am inclined to believe that they did run these tests and found not to provide support for their hypotheses.

There is also a question of whether we should be using .05 as our criterion for statistical significance with these results. (I won’t get into the separate discussion of whether we should be using statistical significance as a way of determining scientific truth at all; that ship has sailed, and until it voluntarily returns to port, we are where we are.) Towards the bottom of the left-hand column of p. 8, we read:
Finally, due to the sample size there was no correction for multiple comparisons; however, we aimed to limit the number of statistical comparisons by pre-selecting the cytokines to measure at the different stages of the study.

It’s nice that the authors pre-selected their predictors, but that is not sufficient. If (as seems reasonable to assume) they also tested the differences between the groups at eight or 12 weeks into the treatment, and found that the results were not significantly different, they should have adjusted their threshold for statistical significance accordingly. The fact that they did not have a very large sample size is not a valid reason not to do this, so I am slightly perplexed by the term “due to” in the sentence quoted above. (The sample size was, indeed, very small. Not only were there only 55 people in total; there were only 18 people in the condition of principal interest, displaying the “CFS-like phenotype”. Under these conditions, any effect would have to be very large to be detected reliably.)

Conclusion

I don’t find Russell et al.’s study to be very convincing. My guess is that different cytokine levels do not predict fatigue in either hepatitis C/IFN-α patients or ME/CFS patients, and that the purported relation between cytokine levels at four weeks into the IFN-α treatment and subsequent fatigue may well just be noise. In terms of explaining how ME/CFS begins, let alone how we might prevent or cure it, this study may not get us any closer to the truth.