By David Tuller, DrPH
I have written many posts about BMJ Open’s 2011 school absence study,
which reported that school absence records could be useful in identifying children with chronic fatigue syndrome. However, for reasons not yet adequately explained, the investigators exempted the study from ethical review on the grounds that it qualified as “service evaluation.” To support the claim, they cited a 2007 letter from the regional research ethics committee (REC), which was unrelated to the study in question.
Studies involving data from human subjects are often categorized as either “service evaluation” or “research.” Those in the former group are generally supposed to evaluate the provision of health care or other services through analyses of anonymous data; they are exempt from ethical review. If a study is testing a hypothesis, produces generalizable conclusions and/or involves data collected from participants known to the investigators, it falls under the category of “research,” not “service evaluation,” and requires ethical review.
The school absence study was conducted by investigators from the University of Bristol. It featured a formal hypothesis, generalizable conclusions, and in-person collection of data from participants. It was clearly research,” as understood by the Health Research Authority, the arm of UK the National Health Service that serves as the arbiter of research ethics. The study should never have been defined as service evaluation and published without ethical review. Yet BMJ Open has repeatedly defended the decision to publish it.
As it turns out, the 2007 REC letter has been cited to exempt at least ten other studies from ethical review as service evaluation. (In some cases, the claimed exemption was only for part of the study.) Perhaps exemption from ethical review could be justified in some instances. But given the Bristol investigators’ inappropriate decision to cite the 2007 REC letter in exempting the school absence study from ethical review, further scrutiny of these studies would seem to be warranted. (A smart source introduced me to the issues involving the 2007 REC letter.)
*********
Here’s the background:
In 2007, the North Somerset and South Bristol REC approved a request to add questionnaires to those already being used during pediatric care at the Bath CFS/ME clinical service. At the time, the clinical service was conducting assessments with several questionnaires at entry and at twelve months. The REC application sought approval for the Bath CFS/ME clinical service to add further assessments at six weeks and six months, arguing that this would be useful for measuring and improving the delivery of care.
In the documents, the title of the research project associated with the application was: “What happens to children with CFS/ME? The study of a longitudinal cohort of children who access a paediatric CFS/ME service.” After reviewing the application, the North Somerset and South Bristol REC sent a letter dated May 1, 2007. Here’s the operative phrasing from this letter: “Members [of the REC] considered this project to be service evaluation. Therefore it does not require ethical review by a NHS Research Ethics Committee or approval from the NHS R&D office.”
The 2007 REC letter referred to “this project”–i.e., the expanded schedule of pediatric assessments at the Bath CFS/ME clinical service. It did not present itself as a precedent for avoiding ethical review for future, as-yet-undescribed projects. Yet that is how members of the Bristol team of investigators seem to have used it.
Besides the school absence study, here are the other papers (in no particular order) that have cited this 2007 REC letter to claim exemption from ethical review:
https://www.ncbi.nlm.nih.gov/pubmed/19452195
http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-11-217
http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-11-308
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3665909/
https://www.ncbi.nlm.nih.gov/pubmed/23619200
http://bmjopen.bmj.com/content/5/10/e008830.full
https://www.ncbi.nlm.nih.gov/pubmed/27655658
https://www.ncbi.nlm.nih.gov/pubmed/19001477
https://www.ncbi.nlm.nih.gov/pubmed/26800634
https://hqlo.biomedcentral.com/articles/10.1186/s12955-018-1028-2
Some of these studies involved data from adults, so it is hard to understand why they would be considered exempt from ethical review based on an REC letter related to routine pediatric questionnaires. Five of the exempted studies were published in BMJ journals–three in Archives of Disease in Childhood and two in BMJ Open, including the school absence study. This doesn’t surprise me. For almost a year, I have been criticizing both journals for lax oversight.
In the case of Archives of Disease in Childhood, our ongoing discussion involves an unrelated study from members of the same team of investigators–a trial of the Lightning Process as a pediatric treatment for children with chronic fatigue syndrome. This trial violated BMJ’s strict policy on prospective registration but is still in the literature. In the case of BMJ Open, the issue has been the school absence study and its lack of ethical review. That study helped trigger my interest in how the same 2007 REC letter was used to exempt other research from ethical review.
**********
Besides the BMJ Open school absence paper, BMJ published four more studies that specifically cited the 2007 REC letter. None seem to meet the criteria for service evaluation of pediatric care.
The second BMJ Open paper on the list involved data from adults as well as children and included this explanation of the finding of the 2007 REC letter: “The North Somerset and South Bristol Research Ethics Committee decided that collection and analysis of CFS/ME patient data constituted service evaluation and did not require ethical review by a NHS Research Ethics Committee or approval by NHS Research and Development offices.”
This expansive reading of the 2007 REC letter would seem to go beyond the meaning and intent of the letter’s finding that “this project”–the collection of specific pediatric data for a longitudinal cohort study–qualified as “service evaluation.” Ethics declarations from other papers in the group use similarly broad terms in describing the 2007 REC letter to exempt themselves from ethical review on the grounds of service evaluation.
The three studies published in Archives of Disease in Childhood and exempted from ethical review on service evaluation grounds do not seem to have focused on the evaluation of pediatric services. Instead, they presented generalizable conclusions about children with the illness.
One of the studies concluded this, for example: “Depression is commonly comorbid with CFS/ME, much more common than in the general population, and is associated with markers of disease severity. It is important to screen for, identify and treat depression in this population.” Studies with generalizable conclusions are normally categorized as “research” and require ethical review.
It is unclear why a narrowly written 2007 REC letter about a particular pediatric research project at one clinical service seems to have been treated as something of an invitation to bypass ethical review in a range of pediatric and adult studies. Bristol University should assess whether each study in this group was or was not appropriately exempted from ethical review based on the finding from the 2007 REC letter. The university should also determine whether members of its faculty commonly use this method to exempt their studies from ethical review.
For its part, BMJ has an independent responsibility to explain why it published multiple studies that exempted themselves from ethical review on questionable grounds, all citing the same 2007 REC letter. Editors need to ask themselves some tough questions. Here are a few:
*What lapses in procedures for assessing and reviewing manuscripts might have allowed these studies to slip through? What red flags were missed?
*What is current policy about fact-checking ethics declarations, especially when they are used to exempt studies from ethical review?
*In the case of the 2011 school absence study, the abstract itself provides sufficient evidence to determine that this was “research” requiring ethical review and not “service evaluation.” Why doesn’t BMJ Open apologize for making an error in judgement and address the issue honestly?
*What steps can BMJ take to prevent such potential embarrassments in the future and to bolster public confidence in the integrity of its editorial decision-making?
If people like you weren’t watching them, they could continue to do bad science and bad statistics and bad exemption – with the appearance of proper work.
Please keep their feet to the fire – this is unconscionable. As a scientist, I’m appalled. As a patient, it just feels like the usual.
I had no idea that the University of Bristol’s obvious breach of research ethics in the School Absence Study is part of a much larger research malpractice involving at least some ten published papers.
There are aspects of the School Absence Study that relate to how genuine or free consent can be in a situation where the parents/guardians are facing possible criminal charges relating to their children’s non school attendance, which make ethical consideration of this research project particularly important. This is also even more essential in a situation where many of the children consequently diagnosed with ME/CFS were likely to be subjected to a highly controversial exercise therapy that may for some result in worsening of their symptoms or even be permenantly disabling, what many would regard as medically sanctioned child abuse. In this situation anything that could be perceived as potentially blocking participants’ right to refuse treatment is highly unethical.
One wonders whether the failure of the Unversity of Bristol and the BMJ stable of publications to address the obvious misconduct in this one study, when it was repeatedly pointed out them, is due to their profound lack of understanding and incompetence or a deliberate attempt to cover up this much larger scandal. Either way such gross incompetence or deliberate malfeasance in a University and the BMJ group of journals, both of international importance, must surely now be investigated by an independent body.
Not all the studies in this list were published before you brought this issue into the public domain, before both Bristol University and the BMJ, at least, were made aware of the problem.
How can this continue to happen? This Bristol research group is engaged in major research projects involving significant sums of money with implications for the health and well being of not only research subjects but also future patients and children subjected to health policy drawing in this research. How can this continue to be happening?
Pingback: Trial By Error: How to Avoid Ethical Review – Virology
This is appalling. Thank you David for your persistence in scrutinising these studies.
In my opinion, school absence is not a valid outcome, anyway, at least not in the UK. There is enormous pressure to attend school, and there is greater pressure to attend during GCSEs, for example. Parents are subject to intimidation and criminal prosecution if their children do not attend school, even when there are valid medical reasons. Schools vary in their environment, e.g. primary schools are much smaller and more pastoral than secondary schools, and schools vary in their provision of special needs. For a child with ME, whether they attend or not will have a lot to do with the type of school they attend and how much support they get from the school and local authority, and can’t be relied upon as a measure of illness.
Thank you David,
Those are great questions.
It’s really an insult to science for people to discuss PACE as if this were a flawed CFS study when fatigue had nothing to do with why this syndrome was coined.
It makes it sound as though doing a “CFS” fatigue study is less than totally objectionable and automatically illegitimate by virtue of not being what the CFS syndrome was originally for.