By David Tuller, DrPH
Let me say this directly: It is way past time for Archives of Disease in Childhood to consign the pediatric Lightning Process study to the trash bin.
As I have pointed out repeatedly, the Bristol University investigators recruited more than half their participants before trial registration, swapped outcome measures based on the early results, and then failed to disclose these critical details in the paper. More than a decade ago, all the major medical journals banned the publication of any studies in which recruitment pre-dates trial registration. For that reason alone, Archives of Disease in Childhood, a BMJ journal, should not have published it in the first place.
In addition to retracting the paper, the journal should offer a full and transparent account of what went wrong. How did the paper pass peer review in the first place? Why has it taken so long for Archives to clean up the mess? What changes has Archives implemented in its review processes, if any?
Beyond the publication realm, the appropriate authorities need to review the conduct and reporting of the Lightning Process study as a case of possible research misconduct. The investigators’ choices appear to meet one or more thresholds for what is defined in ethical codes as research misconduct–such as withholding information that would significantly impact interpretation of the results. Given the known facts, an inquiry into possible research misconduct is warranted. It is hard to understand how members of an experienced team of investigators from Bristol University could fail to recognize the implications of what they did.
**********
Whatever. In the meantime, I have been keeping up with my letter-writing. On Saturday, I sent the following letter to Dr Nick Brown, the editor-in-chief of Archives of Disease in Childhood, and cc-d some others:
Dear Dr Brown,
On June 3, 2018–that is, a year ago–you responded to a letter I had sent to people involved in the development of new ME/CFS guidelines under the auspices of the National Institute for Health and Care Excellence. In my letter, I had alerted them to serious methodological and ethical violations in a high-profile 2017 paper in Archives of Disease in Childhood, a BMJ journal. The paper, from investigators at the University of Bristol, was called “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial.”
Along with people involved in the NICE process, I had cc-d Dr Fiona Godlee, BMJ’s editorial director. You indicated in your e-mail that Dr Godlee had passed on my letter to you. You assured the various recipients that Archives would “respond fully when ready to the concerns I had raised.
Later that month, Archives posted an editor’s note” that acknowledged the concerns, explained that the investigators had provided clarifications, and indicated that the matter was under editorial consideration. The editor’s note was clearly intended as an interim response, not the promised full response. For unexplained reasons, it was not visible from the paper itself but was posted somewhere in what I have referred to as the editorial equivalent of Siberia.
In April of this year, Current Opinion in Pediatrics published a review called “Child and adolescent chronic fatigue syndrome/myalgic encephalomyelitis: where are we now?” This review highlighted the Lightning Process as effective, based solely on the Archives paper. Presumably, the review authors did not notice the obscurely located editor’s note.
(I presume the review authors did not notice the editor’s note because anyone reading it would likely have become alarmed about the validity and reliability of the paper’s reported findings. Certainly such readers would likely have recognized that a decision to cite the Archives findings authoritatively could come back to haunt them.)
The new review in Current Opinion in Pediatrics could impact ongoing public health policy decision-making involving a highly vulnerable population–children with a debilitating and stigmatizing illness. It is therefore urgent to finally address the situation in order to safeguard the medical literature and prevent further dissemination of information based on flawed science. While Archives continues to deliberate the fate of the published paper, the now-appointed NICE committee has been holding meetings to debate and develop the new ME/CFS guidelines.
I have noted repeatedly that it takes half an hour, if that, to examine the relevant trial documents and ascertain what happened. Let’s recap. The investigators recruited more than half the participants before registration, in the process violating a strict policy that BMJ professes to observe. They then swapped outcome measures based on data from these early participants, which allowed them to present positive rather than null results for their reported primary outcome. Then they failed to mention the pre-registration recruitment and the mid-trial outcome-swapping in the paper itself; of course, including these details would have likely doomed the chances of publication in a major journal.
Dr Brown, to those who have been waiting patiently on your year-old promise to respond fully when ready, can you explain why Archives is not yet ready to respond fully, given the prolonged period of editorial consideration? Will you now commit to a deadline by which Archives can be ready to respond fully to the plain set of facts outlined above?
In seeking to expedite resolution of this public health issue and protect children from exposure to recommendations arising from this unacceptable piece of research, I have cc’d the following: Four physicians involved with the NICE committee, along with two patient representatives; Dr Godlee; Dr Terry Segal, the senior author of the review that, to its misfortune, cited the Archives paper uncritically; Sue Paterson, director of legal services at University of Bristol; two members of Parliament who have expressed concerns about the poor quality of much ME/CFS research, along with a parliamentary aide; and Professor Chris Ponting, vice chair of the UK CFS/ME Research Collaborative, a position he assumed following the tenure of the Lightning Process trial’s lead investigator.
Thank you for your attention to this matter.
Best–David
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
Comments are closed.