By David Tuller, DrPH
I sent the following e-mail today to Dr Nick Brown, the editor-in-chief of Archives of Disease in Childhood, the journal that published the Lightning Process study a year ago. I cc’d Dr Fiona Godlee, editorial director of BMJ, which publishes Archives.
**********
Dear Dr Brown,
As you know, I have been requesting since January that Archives of Disease in Childhood address the methodological and ethical lapses that I have documented in a study published last year. The study was a clinical trial of the Lightning Process in children with chronic fatigue syndrome, otherwise known as myalgic encephalomyelitis, CFS/ME or ME/CFS.
As I have noted, the study has provided Lightning Process practitioners with an opportunity to cite the Archives paper as evidence that this woo-woo intervention is effective. Worried parents are undoubtedly spending money they don’t have in a desperate effort to help their children. Given that this clinical trial involves an extremely vulnerable demographic group, the journal’s inability or unwillingness to resolve the matter promptly is disturbing. It creates the impression that Archives and BMJ are more concerned about reputational damage than about ensuring that children receive medical care based on legitimate and robust research.
Most recently, I have criticized an editor’s note that you posted on June 19th, almost four months ago. First, this editor’s note is not directly linked to the article itself but is stashed in the responses section of the site. That means no one reading the article, including parents of children with ME/CFS, would have any idea that serious questions have been raised about the trial’s conduct unless they happened to click on the responses link.
Second, while the editor’s note appears to acknowledge the accuracy of the concerns I have documented, it provides no deadline for completion of the process of what is referred to as editorial consideration of the matter. Since the facts are easy to ascertain, further delay in addressing the issue is pointless and unjustifiable.
To recap: In this study, 56 of the 100 participants were recruited as part of a feasibility study before the registration of the full trial. At the same time as the investigators received permission to extend the feasibility study into the full trial, they also, inexplicably, received permission to swap primary and secondary outcomes. In doing so, they ensured that they could report positive rather than null findings for the full trial’s primary outcome; not surprisingly, the revised primary outcome of self-reported physical function at six months had positive results while the original primary outcome of school attendance at six months had null results.
BMJ has a strict policy against publishing papers in which participants were recruited before trial registration. I assume that, had anyone at the journal read the relevant trial documentation, the paper would never have been published in the first place. I have not come across any exemptions in this BMJ policy for feasibility trials that have been extended into full trials, much less trials in which outcome swaps occurred after more than half the participants had provided data.
Moreover, the full trial was registered as prospective, which it obviously wasn’t. And the paper itself was written as if the trial were prospective. In other words, the investigators failed to disclose the critical fact that 56 % of the participants were recruited before trial registration and that primary and secondary outcome measures were swapped on the basis of the early findings. Given these obvious breaches of core BMJ principles, what argument could there be against retraction of the study?
I must raise two additional critiques. First, the Lightning Process study was an unblinded trial relying on subjective outcomes. In other fields of medicine, it is widely accepted that this type of study design is likely to generate biased results. That is especially so in the case of an intervention like the Lightning Process, which encourages participants to believe that acknowledging the reality of their illness can impede recovery. Therefore, even absent the documented flaws in the conduct of the trial, the findings would have been uninterpretable.
Second, in addition to the positive findings reported for the subjective primary outcome of physical function, the authors also reported positive results for the secondary outcome of self-reported school attendance at 12 months–that is, in contrast to the null results for self-reported school attendance at six months. Yet in both the feasibility and full trial protocols, the investigators promised to obtain official school attendance records to vet the accuracy of self-reported school attendance. Findings from these official school attendance records were not included in the full trial paper, nor did the investigators explain this omission.
There are only two possible reasons: Either the investigators did not collect this data, or they collected it and found that the official school attendance records did not match self-reported school attendance. Neither of these possibilities reflects well on the choices made by the research team.
How could such violations occur? The Lightning Process study has nine co-authors. Perhaps these missteps reflect some sort of honest error. But that would itself be cause for alarm. If true, it would certainly raise concern that other studies from the same investigators might also include similarly egregious mistakes. The only conclusion to be drawn would be that none of the nine co-authors were aware that swapping outcomes midway through a trial is unacceptable because it would bias the results. It would also mean that none of the nine co-authors understood that it was necessary to include these salient details about the trial methodology in the final report.
Whatever the reason, the presence of these flaws suggests insufficient rigor not only in the conduct and reporting of the study but in Archives’ own peer review and editorial processes. Going forward, I would strongly advise you to make sure that peer reviewers and editors access and scrutinize all relevant documentation when determining whether a paper is suitable for publication in a prestigious venue like Archives of Disease in Childhood. Otherwise the journal is likely to experience more embarrassing episodes of this nature.
Beyond the issues with the trial itself, it is worth reiterating that Phil Parker, the creator of the Lightning Process, has previously taught people how to heal others through the use of Tarot, spiritual guides, and related modalities. The following biographical note is from a website promoting one of his courses:
Phil Parker is already known to many as an inspirational teacher, therapist, healer and author. His personal healing journey began when, whilst working with his patients as an osteopath. [sic] He discovered that their bodies would suddenly tell him important bits of information about them and their past, which to his surprise turned out to be factually correct! He further developed this ability to step into other people’s bodies over the years to assist them in their healing with amazing results. After working as a healer for 20 years, Phil Parker has developed a powerful and magical program to help you unlock your natural healing abilities. If you feel drawn to these courses then you are probably ready to join.
Dr Brown, the ongoing delay in dealing with the Lightning Process study is unacceptable. Since this trial is likely to be having an impact on the treatment of children with this devastating illness, it is imperative to conclude Archives’ prolonged period of editorial consideration. The journal must inform the public with all due speed that the findings as reported in the paper were not valid and should not be used as the basis for clinical decision-making in pediatric cases of ME/CFS.
Thank you for your prompt attention to these concerns.
Best, David
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
Comments are closed.