By David Tuller, DrPH
Coda Story is an excellent news organization focused on international stories related to the misuse of science and technology, among other topics. Today, it published a piece of mine about the training program called the Lightning Process. Sites devoted to the Lightning Process are full of tales of recovery from prolonged illness. I included one such account in my Coda article along with accounts from others who reported suffering severe relapses after the training. The article also points out that the scientific claims cannot withstand scrutiny.
To be clear, I’m not telling people whether they should or should not do the Lightning Process. If someone checks it out and thinks it could help them, that’s their call. My concern is that it is promoted as scientific, when it is not. And despite the glowing testimonials, it is easy to find people who say, in contrast, that they experienced serious harms. Their stories are not posted on Lightning Process sites but nonetheless deserve to be shared as well.
While working on the story, I sent an e-mail with a few questions to Lightning Process founder Phil Parker, a British osteopath and psychologist. Within a few days, he responded with a gracious note, along with his answers and a list of supporting references. I have included all the references below, so others can make their own assessments of their merit.
In the Coda story, I mention these references once. In relation to Parker’s statement that the “vast majority” of Lightning Process participants “achieve good and lasting change,” I indicate that the references do not “reasonably support this conclusion.” Obviously, the precise definitions of “vast majority” and “lasting” are up for grabs. But we don’t need to overcome these linguistic challenges to demonstrate that the claim is not grounded in the data.
First, many of the references are for “unpublished” research, that is, not published in a peer-reviewed journal. I’m a journalist as well as a public health academic, so of course I believe in the value of non-peer-reviewed investigations. And I know very well from my own work on this project that peer review is not much of a barrier in many cases. Even so, the issue here is the credibility and robustness of Parker’s sweeping claim of the Lightning Process’ prolonged effectiveness, and it is fair to expect a certain level of, you know, actual documentation. The forwarded references do not meet the required level.
For example, the unpublished 2010 Finch survey reported on the responses of Lightning Process participants when they were asked about the experience right after the end of the program. Here is the salient sentence: “Of the 1092 people who said they had CFS/ME 888 (81.3%) reported that they no longer had CFS/ME by day 3 of the LP course.”
Perhaps 81.3% could be construed as “the vast majority.” But responses at the end of the training program obviously do not represent “lasting change” and might have no relationship to the responses that would be obtained a week or ten weeks later. It should be noted that in my Coda story I included the accounts of two people who would also have offered positive reviews right after the training but crashed badly within months.
Another unpublished Finch study, from 2013, included data for participants extending out three months after the Lightning Process training. At baseline, there were 205 participants. Between baseline and six weeks, the averages of responses to questionnaires on a range of functioning measures rose. Between six weeks and three months, the averages leveled off and hovered in the same general range.
The authors interpret this as an indication that the Lightning Process produces improvements, especially in the first six weeks. But that is hard to tell from the data. Unfortunately, the authors do not indicate how many people responded to questionnaires at six weeks and three months; they do indicate that “missing data” were excluded from the analyses.
This methodological shortcoming renders the analysis of questionable value. If an unknown number of participants did worse and dropped out and the averages at six weeks and three months are drawn only from those remaining, the apparent improvements could be a statistical illusion, an artifact of assessing a shrinking but healthier pool. We simply can’t tell from what is presented.
Moreover, this study provides data only about the averages, or the mean values. Averages of patients’ scores have some interest but they tell us nothing about how many people get better and how many get worse, and by how much, which is what doctors and patients really want to know. Averages can be dramatically affected by outliers. If you averaged my income with Bill Gates’ income, we’d both look like we have billion-dollar incomes. We don’t.
In short, this study also does not support any assertion about the “vast majority” of participants; we don’t even know if the “vast majority” of the 205 participants filled out their three-month questionnaires. And three months is too short a time-frame to be considered “lasting,” by my understanding of the word.
Interestingly, Table 1 indicates that “data for 6 and 12 months is [sic] currently in collection.” Presumably these longer-term data were “in collection” with the intention of calculating longer-term outcomes. Parker did not provide a follow-up study including these later results, which would have been helpful in making his case for “lasting change.”
Let’s see, what else? The Hagelsteen & Moen Reiten article was published not in a peer-reviewed journal but in something called Dagens Medicin, a news update for people working in health care. The article is a report about a dozen kids with chronic headaches who received the Lightning Process. A year later, nine reported they were better. I think we can all agree that this is also not sufficient evidence for stating that the “vast majority” of the 25,000 Lightning Process participants have achieved “good and lasting change.”
That’s about it. The other references are equally unconvincing.
The Landmark et al article, for example, noted that we know too little about the Lightning Process and called for more research. And Parker, interestingly, cited the 2013 protocol, Crawley et al, for the one clinical trial that has been conducted to date, However, he did not cite the 2017 report of the actual results. Perhaps that is because he knows the study has been slapped with a 3,000-word correction triggered by my investigation of its methodological and ethical violations, and that the reported findings are therefore suspect.
Presumably, Parker made what was his best case for the Lightning Process last year in a paper that he and two colleagues called a “systematic review of the evidence base.” Brian Hughes, a professor of psychology at NUI Galway, was not impressed with this presentation, as he noted in a post on his blog, The Science Bit. Hughes’ entire post is well worth a read, but here’s an excerpt:
This so-called “systematic review” is surely one of the most atrocious academic papers that I have ever had the misfortune to read. It isn’t even a “systematic review”. Rather, it is a self-serving pseudostatistical jargon-filled waffle-fest, utterly untroubled by even the tiniest smidgen of scholarly objectivity. It is, in fact, deeply depressing.
**********
Bibliography (References sent to me by Phil Parker)
Crawley, E., Mills, N., Hollingworth, W., Deans, Z., Sterne, J., Donovan, J., Beasant, L., & Montgomery, A. (2013). Comparing specialist medical care with specialist medical care plus the Lightning Process® for chronic fatigue syndrome or myalgic encephalomyelitis (CFS/ME): Study protocol for a randomised controlled trial (SMILE Trial). Trials, 14, 444. https://doi.org/10.1186/1745-6215-14-444
Finch, F. (2010). Snapshot Survey 2018 for clients with CFS.ME.pdf. http://philparker.org/wp-content/uploads/2010/07/Snapshot%20Survey%202018%20for%20clients%20with%20CFS.ME.pdf
Finch, F. (2013). Outcomes Measures Study. doi.org/10.13140/RG.2.2.29818.24002
Finch, F. (2014). MS Proof of Concept Study. doi.org/10.13140/RG.2.2.26462.79686
Hagelsteen, J. H., & Moen Reiten, I. M. (2015). Evaluation of a treatment strategy. Dagens Medicin. http://www.dagensmedisin.no/artikler/2015/10/12/evaluering-av-en-behandlingsstrategi/
Kristoffersen, A. E., Musial, F., Hamre, H. J., Björkman, L., Stub, T., Salamonsen, A., & Alræk, T. (2016). Use of complementary and alternative medicine in patients with health complaints attributed to former dental amalgam fillings. BMC Complementary and Alternative Medicine, 16, 22. https://doi.org/10.1186/s12906-016-0996-1
Landmark, L., Lindgren, R. M. B., Sivertsen, B., Magnus, P., Sven Conradi, Thorvaldsen, S. N., & Stanghelle, J. K. (2016). Chronic fatigue syndrome and experience with the Lightning Process. Tidsskrift for Den norske legeforening, 136(5), 396–396. https://doi.org/10.4045/tidsskr.15.1214
Parker, P. (2011). Dû: Unlock your full potential with a word. Nipton Publishing.
Parker, P. (2012). An introduction to the Lightning Process®: The first steps to getting well. Hay House.
Parker, P. (2013). Get the life you love, now: How to use the Lightning Process® toolkit for happiness and fulfilment. Hay House.
Parker, P. (2020). LP Protocol 2020. https://doi.org/10.13140/RG.2.2.22761.72801
Parker, P., Aston, J., & de Rijk, L. (2020). A Systematic Review of the Evidence Base for the Lightning Process. EXPLORE, 1–30. https://doi.org/10.1016/j.explore.2020.07.014
Parker, P., Aston, J., & Finch, F. (2018). Understanding the Lightning Process approach to CFS/ME; a review of the disease process and the approach. Journal of Experiential Psychotherapy, 21(2), 8. https://jep.ro/images/pdf/cuprins_reviste/82_art_2.pdf
Parker, P., Banbury, S., & de Rijk, L. (2021). Self-control or Flourishing? A Thematic Analysis of Experiences of Alcohol Users of the Rediscovery Process. International Journal of Mental Health and Addiction. https://doi.org/10.1007/s11469-021-00520-3
Reme, S. E., Archer, N., & Chalder, T. (2012). Experiences of young people who have undergone the Lightning Process to treat chronic fatigue syndrome/myalgic encephalomyelitis, A qualitative study. British Journal of Health Psychology, 18(3), 508–525. https://doi.org/10.1111/j.2044-8287.2012.02093.x
Sandaunet, A.-G., & Salamonsen, A. (2012). CFE-/ME-pasienters ulike erfaringer med Lightning Process. Sykepleien Forskning, 7(3), 262–268. https://doi.org/10.4220/sykepleienf.2012.0132
Sussex & Kent ME/CFS Society. (2010). Sussex and Kent report. https://lightningprocess.com/research/
Comments are closed.