Trial By Error: Stupid Studies

By David Tuller, DrPH

Boy, it€™s hard to keep up with all the stupid studies coming out! But that€™s no reason not to take a look at a couple of them.

(In both cases, I’m just making a few comments. Much more could be said about each of these studies. You can read more about them on the Science for ME forum here and here.)


In December, Frontiers in Psychology published a paper about a four-day mindfulness-based cognitive behavioral intervention program for CFS/ME being offered in Norway. The authors of the study, two mental health professionals from Bergen, have tested what seems to be a ratatouille of group therapy, exercise, life-coaching, meditation, stress management, writing exercises and positive thinking, something similar, perhaps, to the Lightning Process, although I don€™t know if participants are required to stand inside circles on the ground.

The study will likely be cited, in Norway and elsewhere, as evidence that this four-day program could be effective in reducing fatigue and improving physical function in people with CFS/ME, as the investigators call the illness. Yet the findings are pretty much useless in terms of providing actionable information for developing health policy or clinical recommendations, for a number of reasons.

First, this isn€™t a trial. It is an observational study of 305 patients who have gone through a four-day experience. That€™s it. There is no control group against which to measure the reports of success among these participants. It is possible that a program of many hours of playing Monopoly in an elevator while listening to Adele singing Rolling in the Deep over and over could have produced the same effect. We just don€™t know.

Second, patients were recruited according to the Oxford criteria, six months of disabling, unexplained fatigue. In other words, the sample was likely a mish-mash of people with fatigue for all sorts of reasons. Moreover, not even half the participants were disabled enough to meet the PACE thresholds for trial entry on both the subjective fatigue and physical function scales. So it€™s hard to know what illness many of the participants had, if any. (As in PACE, participants were re-assessed after recruitment to see if they also met the 1994 Fukuda criteria from the US Centers for Disease Control and Prevention.)

Third, all the outcomes were subjective. So, you know, it€™s hard to tell what any of these outcomes mean. The intervention was apparently effective at training people to provide somewhat better responses on questionnaires.

The authors acknowledge some limitations but claim success anyhow. They report that, per their results, the program might be highly beneficial and that 80% of the participants demonstrated statistically significant clinical changes, which were sustained after a year. Unfortunately, the claim that the changes were sustained at the one-year point does not seem to take into account the fact that only 219 of the 305 study participants provided follow-up data.

That€™s a sharp drop-off. Did the 86 who failed to provide follow-up data get worse? Who knows?


Someone keeps funding research by Trudie Chalder. This is hard to understand. Her most recent nonsense is a study called Perfectionism and beliefs about emotions in adolescents with chronic fatigue syndrome and their parents: a preliminary investigation in a case control study nested within a cohort. It was just published in the journal Psychological Health.

The study posits that psychological processes, including perfectionism and beliefs about emotion, are risk factors for the prolonged symptoms associated with CFS. It also posits that these attitudes are transmitted to children through parental influence. In other words, if the parents exhibit high levels of perfectionism, so will the kids, and that creates an environment in which CFS can presumably thrive.

Well, the study findings contradicted the researchers€™ expectations. They found no evidence for a link between personality traits and CFS. In other words, kids with CFS showed no differences in perfectionism and beliefs about emotion than peers with asthma and healthy controls. Moreover, when the kids with CFS were followed up, perfectionism and beliefs about emotions did not account for variances in fatigue and physical function.

In this preliminary study, we did not find evidence for the hypothesis that adolescents with CFS would report higher levels of perfectionism and unhelpful beliefs about emotions than adolescents with asthma or healthy adolescents, write Professor Chalder and her colleagues.

Nonetheless, the study goes out of its way to argue that the absence of evidence does not disprove the existence of such a link, given the study design. This is undoubtedly true. Yet the tortured logic on display comes across as wishful thinking, even desperation. The investigators argue that the experience of illness itself could have forced patients to drop the perfectionist traits they possessed before getting ill.

Here€™s what they write: With this cross-sectional study, which recruited adolescents with CFS post-diagnosis, it is not possible to conclude that perfectionism does not pose a vulnerability factor to developing CFS…It is possible that people are forced to modify their perfectionism as a result of disabling CFS symptoms or that they choose to do so.

In other words, don€™t pay attention to our findings!

That their predominant tendency is to ignore or overlook their own main findings is clear from their conclusion, as expressed in their abstract. The conclusion focuses on a different, not surprising, and relatively minor finding, that children€™s personality traits might reflect those of their parents.

Here€™s what they write: Parental perfectionism and emotion regulation style may contribute to perfectionism in adolescents with CFS. Parental representations could contribute to fatigue maintenance.”

It may or may not be the case that perfectionism in parents contributes to perfectionism in their offspring. But given that the study found no link between perfectionism and CFS in adolescents, any connection between the emotion regulation style of parent and child would seem to be irrelevant to the matter at hand. I suppose it is possible that such parental representations could contribute to fatigue maintenance. But it is unclear why the authors are advancing this claim as their concluding point when the study provides little apparent evidence to support it.

Does anyone actually peer review this stuff?

Comments are closed.

Scroll to Top