By David Tuller, DrPH
*October is crowdfunding month at Berkeley. I conduct this project as a senior fellow in public health and journalism and the university’s Center for Global Public Health. If you would like to support the project, here’s the place: https://crowdfund.berkeley.edu/project/22602
I have written many posts this year about a Norwegian study of cognitive behavior therapy plus music therapy as a treatment for chronic fatigue after acute EBV infection (aka mononucleosis and glandular fever). The study was rife with methodological and ethical flaws and should never have been published.
The investigators presented their research as a feasibility study seeking data to justify funding for a fully powered trial. In fact, it was designed as a fully powered trial, but it failed to meet recruitment goals and produced terrible results. The false description of the study was only one of many issues of concern that several colleagues and I raised in a May 31st letter to the journal and Dr Fiona Godlee, BMJ’s editorial director. (Michiel Tack, a smart patient and advocate in Belgium, had previously written a cogent letter to the journal outlining key issues.)
Last Thursday, Retraction Watch reported that BMJ Paediatrics Open had retracted the initial article and replaced it with a new one. The same day, I received an e-mail from BMJ informing me of the same.
This is not a satisfactory resolution. The new version is still sub-par, even if it presents itself accurately as a randomized trial that ended up being underpowered. Norwegian blogger Nina Steinkopf, who has tracked this issue from the start, has offered her analysis of the new version.
The retraction notice itself is problematic. For one thing, it claims to be transparent but omits mention of the broken peer-review process. I will have more to say about this at another point. For now, below is the message I received from BMJ last Thursday, followed by my response.
Dear Dr Tuller,
Thank you for bringing this publication to our attention.
We undertook an internal review to ensure that the editorial and peer review processes and identified some areas where our process should have been more robust. We agree that the paper should have been presented as an underpowered randomised controlled trial rather than a feasibility study. We agree that the statistical review should have been more thorough. We have retracted the paper and we invited the authors to resubmit their manuscript which has undergone full editorial and peer review. We have now republished their research.
Research Integrity Co-ordinator
Dear Ms Ragavooloo–
Thank you for your much-delayed response. I plan to consult with my colleagues but have a few initial thoughts.
BMJ editors apparently decided months ago that the first version needed to be retracted. Given that, it is troubling that the paper remained posted without notice until the revision was ready to be published. Can you explain why BMJ Paediatrics Open kept an invalid paper in the literature for any period of time after editors had recognized it should not have been published in the first place? For BMJ to not inform readers that a paper they are reading is going to be retracted in the future does not suggest a serious commitment to research integrity.
It should be noted as well that, among other issues, the new version still includes post-exertional malaise as an outcome, even though this measure was not mentioned in the protocol or statistical analysis plan–as my colleagues and I pointed out in our initial letter of concern. At a minimum, this suggests once again that the peer-review process was deficient. (Unless BMJ Paediatrics Open has a policy specifically allowing investigators to add outcomes that were not mentioned in the protocol or statistical analysis plan.)
Was this new version actually peer-reviewed? If so, do you plan to post the reviews, per BMJ Paediatrics Open policy? If not, why not?
As you know, in the first round BMJ editors did not notice or care that one of the two reviewers acknowledged not having read “beyond the abstract.” Given that embarrassing history, how can readers feel assured that the same thing did not happen again?
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley