By David Tuller, DrPH
Venture capitalist Robert Paull is listed on the Mahana Therapeutics website as the company’s co-founder and CEO. I have no idea how he got involved with this group of UK researchers and King’s College London. Given my own experiences, I would certainly have advised him against getting in bed with them, had he asked me–which he didn’t.
As it is, Mr Paull’s company has now licensed a product that generates minimal and transient reported benefits in reducing symptom severity in irritable bowel syndrome–yet is promoting the product based on its purported success in long-term reduction of symptom severity. I pointed out this dilemma in the letter I sent Mr Paull earlier today.
Dear Mr Paull–
I am a senior fellow in public health and journalism at UC Berkeley’s Center for Global Public Health, which is part of the School of Public Health. I write frequently about research on illnesses in the category of so-called “medically unexplained symptoms.” Much of my work appears on Virology Blog, a science site hosted by Vincent Racaniello, a professor of microbiology at Columbia. (I have cc’d Dr Racaniello on this e-mail.)
I have recently written about Mahana Therapeutics’ licensing deal with King’s College London, which was announced last month. The deal is for a web-based cognitive behavior therapy program for treating irritable bowel syndrome, which is apparently slated to be Mahana’s first product. I initially critiqued the ACTIB study that road-tested the web-based program in this post, following up with two posts–here and here–about the licensing deal itself.
It is true, as you are quoted as saying in the press release about the deal, that there are “extensive clinical data” about the web-based program. Unfortunately, both the press release and the Mahana website convey seriously misleading information about what these clinical data from the ACTIB study mean.
Mahana’s press department has not responded to three e-mail messages I have sent in my efforts to seek responses to my concerns. I have also reached out to Professor Rona Moss-Morris, ACTIB’s co-lead investigator, and some of the gastroenterology advisors listed on the Mahana site.
According to the study, the web-based program demonstrated very modest reported benefits in reducing symptom severity on an outcome measure called the IBS-SSS. At 12 months, the mean score of patients in the web-based arm on this measure was 35.2 points lower than the mean score for those who received treatment-as-usual. (A reduction of 50 points or more on the IBS-SSS is considered clinically significant; a reduction of less than 50 points is not considered clinically significant.) At 24 months, the 12.9-point difference in mean scores between the two groups was not statistically significant.
In its press release about the licensing deal, Mahana described these improvements in symptom severity as “substantial” and “durable.” On its website, Mahana is promoting them as “dramatic and potentially game-changing for patients.” I do not think these statements accurately reflect the fact that the difference between the mean scores for the web-based and treatment-as-usual groups was less than the designated threshold for reaching clinical significance, and in fact was non-existent at long-term follow-up.
Moreover, the website notes that, at 12 months, 66% of those in the web-based arm had “clinically meaningful reduction” in IBS severity–in this case, that means their IBS-SSS scores were reduced by 50 or more points. Yet fhe website fails to mention that 44% of participants who received treatment-as-usual and provided data at 12 months also achieved that threshold.
Do you agree that the omission of such an important detail is likely to create the impression among readers that all the reported improvements should be attributed to the web-based program? To be fully transparent, do you believe the website should also mention that 30% of the trial participants did not provide these 12-month responses, so their IBS-SSS outcomes are unknown?
The IBS-SSS was the only instrument in the study specifically designed to assess IBS symptoms rather than more generic domains also assessed by self-reported questionnaire, such as depression and social adjustment. Modestly positive reports on these additional measures might be expected from a round of CBT and should not be taken as signs of successful treatment of IBS or remission of IBS symptoms. In any event, Mahana is touting the IBS-SSS results themselves as evidence of robust impacts–even though these results clearly do not provide such evidence.
Since I have criticized Mahana’s promotional approach, I would be happy to post any response you want to make on Virology Blog–at full length and without editorial interruption from me. As well, here are some other specific questions:
1) Do you really believe the ACTIB findings represent “substantial,” “durable,” “dramatic,” and “potentially game-changing” reductions in symptom severity? If so, what data have you seen that can support these claims?
2) Do you think that this sort of rhetoric, which would seem likely to induce hopes of improvement in symptom severity that go beyond the study findings, is acceptable and appropriate when marketing medical products?
3) Can you explain why Mahana decided to license a program that produces such minimal and transient benefits in reducing symptom severity, only to promote the program based on its purported success in long-term reduction of symptom severity?
Thanks much! I look forward to hearing from you. To keep Virology Blog readers informed of my reporting efforts, I plan to post this letter.