By David Tuller, DrPH
I have recently written a few posts–here, here and here–about a study in BMJ Paediatrics Open that appears to be marred by multiple methodological and ethical problems. This is certainly not a one-time occurrence when it comes to BMJ journals. Last week, I sent a letter to the study’s senior author inviting him to send me his response for posting in full on Virology Blog. I have not heard back.
BMJ Paediatrics Open features an open–that is, non-anonymous–peer review process. Continuing my efforts to understand how a fully powered randomized trial that failed to meet expectations was published as if it had always been intended as a feasibility study undertaken to generate support for a fully powered randomized trial, I have sent the following letter to the one of the two peer reviewers who actually read the paper. (One of them announced bluntly that he had not read “beyond the abstract.”)
Dr Maria Loades is a lecturer in the psychology department at the University of Bath. I sent her the following letter this morning. (Although I wrote that I was cc-ing Professor Racaniello, I actually forgot to add him, so I forwarded the letter to him afterward. Just to clarify.)
**********
Dear Dr Loades-
I am a journalist and academic fellow at the School of Public Health at the University of California, Berkeley. I frequently write about research in the domains of CFS, ME and other so-called medically unexplained symptoms. Much of my work appears on Virology Blog, a science site hosted by Professor Vincent Racaniello, a microbiologist at Columbia University. (I have cc’d Professor Racaniello on this e-mail.)
I have written three Virology Blog posts this month that raise questions about a study published recently by BMJ Paediatrics Open called €œCognitive behavior therapy combined with music therapy for chronic fatigue following Epstein-Barr virus infection in adolescents: A feasibility study.” The three Virology Blog posts are here, here and here.
Last December, you peer reviewed a draft of this study as part of the journal’s open peer review process. At that point, the title referred to the study as €œan exploratory randomized trial.€ While you praised the paper overall, you expressed confusion about the original goal of the investigators. Here’s what you wrote:
“I struggle to understand from the aims of the study and the way the study is described whether this was intended as a feasibility study €“ i.e. to look at feasibility (can this be done?), acceptability (how do participants experience it?) and to give some indication of potential effect sizes to power a future larger scale trial, or whether this was intended as a fully powered trial. Throughout, I think this needs to be clarified for the reader and interpretations/conclusions drawn in light of what the aim was.”
In response, the investigators wrote this: €œThank you. We agree €“ this study should be regarded [as] a feasibility study, and the manuscript has been rephrased accordingly.€
And here is how the published paper described the aim: €œThe aim of the present study was to explore the feasibility of this mental training programme in adolescents suffering from CF after acute EBV infection, and to provide preliminary estimates of effects as a basis for a full-scale clinical trial in the future.€
Dr Loades, your confusion after having read the first draft of the paper was understandable. So was the request for clarity as to the original aim of the research. But instead of providing clarity, the published version offers an account that differs significantly from key trial documents.
As is clear from a review of the trial protocol, registration and statistical analysis plan, the research was designed as a small but fully powered trial, not as a feasibility study, as declared in the published paper. Unfortunately for the investigators, trial recruitment fell below expectations, the CBT-music therapy intervention group experienced high attrition, and the results were disappointing.
Yet the version published by BMJ Paediatrics Open does not describe the study as a fully powered trial that did not meet expectations. Although you suggested that the investigators draw their interpretation and conclusions €œin light of what the aim was,” they appear to have drawn their interpretation and conclusions in light of how they recast that aim after-the-fact.
Dr Loades, did you know that the published version diverged in this significant way from the trial protocol, registration and statistical analysis plan? Are you surprised that the investigators responded to your comments by reporting how they thought the study “should be regarded” after it was done rather than how they originally designed it? If you reviewed and approved the revised version, do you stand by that approval?
I sent a letter last week to the corresponding author to solicit his explanation for this anomaly as well as other perplexing aspects of the research. I have not heard back.
I look forward to hearing from you. For full transparency, I plan to post this letter on Virology Blog.
Best–David
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley
Comments are closed.