By David Tuller, DrPH
As I wrote yesterday, Mahana Therapeutics has recently licensed from King’s College London an “innovative digital therapeutic”, a web-based program delivering a course of cognitive behavioral therapy to patients with irritable bowel syndrome. A page on the Mahana site promoting this web-delivered IBS-CBT program furthers the impression that this deal is steeped mostly in hype.
A January 10thpress release announcing the deal declared the changes in symptoms produced by the web-based program to be “substantial,” which already seemed like an exaggeration. The company’s website has apparently upgraded this observation with its description of the symptom improvements as “dramatic and potentially game-changing for patients.” Wow!
(It should be noted that Professor Rona Moss-Morris of King’s College London, one of the principal investigators of the study that road-tested the web-based program being licensed, has been paid as a consultant by Mahana and also owns stock options. Her ownership of Mahana stock options was not in the disclosure for the study of the web-based program but in her disclosure for a separate trial for people with persistent physical symptoms. Incidentally, that paper about the other trial had to be corrected last year after I pointed out that it included a false claim concerning the estimated costs of so-called “medically unexplained symptoms”–a claim based on a misreading by Professor Moss-Morris and her co-authors of a seminal study in their field of expertise.)
In the published papers, the web-based program was called Regul8. Mahana has re-branded it as Parallel. (It’s possible the academic investigators divided the material into eight chapters specifically so the name Regul8 would work and come across as clever. I don’t know what “parallel” refers to.) And here’s what we’re told about CBT: “For years, CBT has been used successfully to treat health issues like chronic pain, heart disease, and multiple sclerosis.”
Really? I haven’t heard of people recovering from MS through CBT, which is what the sentence would likely mean to many or most readers. I assume Mahana intended to convey the notion that CBT is a helpful adjunct for alleviating some symptoms of MS and other complaints. Sure enough, the footnoted reference for the MS claim does not involve treatment of MS but of the fatigue that can accompany MS.
Did anyone proof-read or vet this website? Mahana’s apparent inability to understand or appreciate the distinction between treating a condition itself and addressing symptomatic relief is alarming for a company that presents itself as being on the cutting edge of science.
In its description of Parallel, Mahana prominently cites two pieces of evidence from the trial I wrote about last week.
- “66% of patients reported significant and clinically meaningful reduction in the severity of their IBS”
This is a misleading claim. It is true that 66 % of those in the web-based CBT arm who responded at 12 months had a reduction in the scores on the IBS Symptom Severity Scale of 50 or more points. (A reduction of fifty or more points is considered clinically significant.) But it is not true that most of those changes can be attributed to the web-based program, which is what the statement appears to imply. Mahana does not mention that 44% of those in the treatment-as-usual arm who reported at 12 months also had a reduction in scores of 50 or more on the same scale. Given those numbers, it seems likely that many or most of the 66 % in the web-based arm could have reported those improvements anyway.
Moreover, the site does not make clear that only 70 % of the study sample provided data at 12 months. We can’t know what the final results for the remaining 30 % would have been. That means we have no idea how those who dropped out from the web-based program arm felt about the intervention or whether it helped them. They were, in epidemiological terms, “lost to follow-up.”
In any event, the study’s main analysis of the IBS-SSS scores used an “intention-to-treat” strategy, which tries to account for this sort of missing data. At 12 months, the mean score for the web-based group was found to be 35.2 points lower than for the treatment-as-usual group, quite a bit less than the 50-point difference that would represent a clinically significant improvement for an individual.
- “On average, reduction in IBS severity was twice that of patients receiving medical care as usual.”
Again true, and again misleading. When improvements are small, improvements that are twice the size are also pretty small. Just because something doubles does not automatically mean the change is of much clinical significance. The more telling statistic is often not the relative difference between groups–the kind cited in the above claim about average reductions of IBS severity–but the absolute difference. In this case, as I’ve already noted, the absolute difference in score between the means of the groups was 35.2 points–well under the 50 points that would represent a clinically significant change for an individual on that scale.
Besides the inaccurate marketing, a couple of other issues raise concerns.
First, is Mahana planning to present Parallel for use with therapeutic input? That’s how it was road-tested. It cannot be assumed that findings from the study would be replicated in the absence of therapist guidance. Nothing in the press release about the licensing deal or on the Mahana website itself suggests that the web-based program will be delivered with some doses of human contact as well, but perhaps it will be.
Second, the Mahana website ignores the multiple reasons to be skeptical of even the unimpressive results reported in the study. These reasons include the possibility of bias in an open-label trial with solely subjective outcomes, the unfortunate decision to inform patients of CBT’s purported effectiveness in treating IBS through information included in the trial manual, and the fact that patients getting the intervention were compared to a group of patients getting what was, for all practical purposes, little better than nothing.
In the end this big licensing deal seems like a lot of noise about very little. But I assume someone somewhere will be making money from it. Nice!
Judging by the website, Parallel is Mahana’s only product. But get ready for more good news down the line! Here’s what we also learn on the site:
Mahana is currently developing a personalized digital treatment for adolescents and adults with Inflammatory Bowel Disease. This product will be tested in a series of research studies and randomized, controlled trials. This treatment will tap into the power of Cognitive Behavioral Therapy (CBT) to help patients living with IBD.
Comments are closed.