Trial By Error: An Open Letter to The Lancet, Two Years On

By David Tuller, DrPH

This morning, Professor Racaniello sent the following e-mail to Richard Horton, editor of The Lancet. The subject heading: “Another open letter about the PACE trial.” He cc’d the three lead PACE investigators and the public relations office at Queen Mary University of London. Virology Blog’s previous open letter to The Lancet about the PACE trial was sent and posted in February, 2016.

**********

Dear Dr. Horton:

In February, 2011, The Lancet published an article called “Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomized trial.” [1] The article reported that two rehabilitative approaches, cognitive behavioural therapy (CBT) and graded exercise therapy (GET), were effective and safe treatments for chronic fatigue syndrome, also often referred to as myalgic encephalomyelitis, ME/CFS and CFS/ME. The PACE study received international attention and has had widespread influence on research, treatments prescribed for patients, and attitudes toward the illness of both the medical community and the public at large.

At the press conference promoting the Lancet paper, one of the lead investigators stated that twice as many participants in the treatment groups got “back to normal,” compared to those in the other study arms. [2] An accompanying Lancet commentary similarly claimed that these “back-to-normal” participants had met a “strict criterion for recovery.” [3]

In fact, we now know that 13 % of the participants qualified at baseline as “recovered” or “within the normal range” for one of the study’s two primary measures, self-reported physical function–even as they were simultaneously classified as disabled enough on the same measure to enter the study. [4] This anomaly, which occurred because the investigators weakened key outcome thresholds after data collection, invalidates any claims that patients “recovered” or got “back to normal.” The overlap in entry and outcome criteria is only one of the trial’s unacceptable methodological lapses.

The treatments investigated in the PACE trial were based on the hypothesis that ME/CFS patients harbor “unhelpful” convictions about having an ongoing organic disease and that the perpetuation of their devastating symptoms is the result of deconditioning. In contrast, a 2015 review from the U.S. Institute of Medicine (now the National Academy of Medicine), reported that ME/CFS is a complex, multi-system illness characterized by neurological, immunological, autonomic, and energy metabolism dysfunctions. [5] The cardinal symptom, noted the review, is a systemic intolerance to exertion; if patients exceed their available energy resources, they can suffer serious and prolonged relapses.

After The Lancet published the first PACE results, ME/CFS patients and advocates immediately pointed out major flaws. But few people outside the field took notice until the science site Virology Blog published a 15,000-word investigation by David Tuller, a public health researcher and journalist at the University of California, Berkeley, in October of 2015. [6] Subsequently, in February of 2016, many of us signed an open letter to The Lancet requesting an independent investigation of the study. [7]

Since then, much has happened:

* In August of 2016, a U.K. tribunal, citing that open letter, ordered Queen Mary University of London to release raw trial data from the PACE study, sought by Australian patient Alem Matthees in a freedom of information request so that he and others could calculate the outcomes promised in the PACE trial protocol. [8]

* Analyses of these data [9], including a study published in BMC Psychology in March [10], have confirmed what has long been argued: The PACE investigators engaged in such extensive outcome-switching that they were able to report dramatically better findings than the null or minimal results obtained under the original measures they promised in their protocol.

* The U.S. Agency for Healthcare Research and Quality (AHRQ) downgraded its recommendations for CBT and GET. [11] This downgrading occurred after the agency removed from its analysis the PACE trial and other studies using overly broad selection criteria that generated cohorts of patients with a grab-bag of fatiguing conditions. And while the PACE trial claimed that GET is safe, AHRQ found that the therapy was associated with more adverse events.

* Last summer, the U.S. Centers for Disease Control abandoned the recommendations that ME/CFS patients be treated with CBT and GET [12], having already removed references to the PACE trial. A couple of months later, the U.K. National Institute for Health and Care Excellence announced that it would pursue a full update of its 2007 guidance, citing concerns about the reliability and validity of the evidence base. [13]

* Earlier this year, a report from the Dutch Health Council recommended that GET should not be used in the Netherlands as a treatment for the illness. [14]

* In March, a group of leading American clinicians who specialize in ME/CFS unanimously agreed that the two PACE treatments are inappropriate and possibly harmful for patients with the illness and should therefore not be prescribed. [15]

Given the worldwide impact of PACE, we urge The Lancet to do what the open letter two years ago requested: commission an independent re-analysis of the individual-level trial data, with appropriate sensitivity analyses, from highly respected reviewers with extensive expertise in statistics and study design. The reviewers should be from outside the domains of psychiatry and psychological medicine and predominantly from outside the U.K. They should also be completely independent of, and have no conflicts of interests involving, the PACE investigators and the funders of the trial.

Thank you for your quick attention to this matter.

Sincerely,

Dharam V. Ablashi, DVM, MS, Dip Bact
Scientific Director, HHV-6 Foundation
Santa Barbara, California, USA
Former Senior Investigator
National Cancer Institute
National Institutes of Health
Bethesda, Maryland, USA

Michael Allen, PhD
Clinical Psychologist (retired)
San Francisco, California, USA

Christopher Armstrong, PhD
Bio21 Molecular Science & Biotechnology Institute
Department of Biochemistry and Molecular Biology
University of Melbourne
Melbourne, Victoria, Australia

James N. Baraniuk, MD
Professor of Medicine
Georgetown University
Washington, DC, USA

Lisa F. Barcellos, PhD
Professor of Epidemiology
School of Public Health
California Institute for Quantitative Biosciences
University of California, Berkeley
Berkeley, California, USA

Lucinda Bateman, MD
Medical Director
Bateman Horne Center
Salt Lake City, Utah, USA

Molly Brown, PhD
Assistant Professor
Department of Psychology
DePaul University
Chicago, Illinois, USA

Robin Callender Smith, PhD
Professor of Media Law
Centre for Commercial Law Studies
Queen Mary University of London
Barrister and Information Rights Judge
London, England, UK

John Chia, MD
Clinician and Researcher
EV Med Research
Lomita, California, USA

Lily Chu, MD, MSHS
Independent Researcher
Burlingame, California, USA

Joan Crawford, CPsychol, CEng, CSci, MA, MSc
Chartered Counselling Psychologist
Chronic Pain Management Service
St Helens Hospital
St Helens, England, UK

Janet L Dafoe, PhD
Child Psychologist in Private Practice
Palo Alto, California, USA

Todd E. Davenport, PT, DPT, MPH, OCS
Professor & Program Director
Department of Physical Therapy
Thomas J. Long School of Pharmacy & Health Sciences
University of the Pacific
Stockton, California, USA
Workwell Foundation
Ripon, California, USA

Ronald W. Davis, PhD
Professor of Biochemistry and Genetics
Stanford University
Stanford, California, USA

Lucy Dechene, PhD
Professor of Mathematics (retired)
Fitchburg State University
Fitchburg, Massachusetts, USA

Simon Duffy, PhD, FRSA
Director
Centre for Welfare Reform
Sheffield, England, UK

Jonathan C.W. Edwards, MD
Emeritus Professor of Medicine
University College London
London, England, UK

Valerie Eliot Smith
Barrister and Visiting Scholar
Centre for Commercial Law Studies
Queen Mary University of London
London, England, UK

Derek Enlander, MD
Clinician in Private practice
New York, New York, USA

Meredyth Evans, PhD
Clinical Psychologist and Researcher
Chicago, Illinois, USA

Kenneth J. Friedman, PhD
Associate Professor of Physiology and Pharmacology (retired)
New Jersey Medical School
University of Medicine and Dentistry of New Jersey
Newark, New Jersey, USA

Robert F. Garry, PhD
Professor of Microbiology and Immunology
Tulane University School of Medicine
New Orleans, Louisiana, USA

Keith Geraghty, MPH, PhD
Honorary Research Fellow
Division of Population Health, Health Services Research & Primary Care
School of Health Sciences
University of Manchester
Manchester, England, UK

Simin Ghatineh, MSc, PhD
Biochemist
London, England, UK

Ian Gibson, PhD
Former Member of Parliament for Norwich North
Former Dean, School of Biological Sciences
University of East Anglia
Honorary Senior Lecturer and Associate Tutor
Norwich Medical School
University of East Anglia
Norwich, England, UK

Mike Godwin, JD
Attorney and Author
Distinguished Senior Fellow
R Street Institute
Washington, DC, USA

Rebecca Goldin, PhD
Professor of Mathematics
George Mason University
Fairfax, Virginia, USA

Alan Gurwitt, MD
Clinician in Private Practice (retired)
Associate Clinical Professor, Yale Child Study Center (retired)
New Haven, Connecticut, USA
Associate Clinical Professor, University of Connecticut Dept of Psychiatry (retired)
Storrs, Connecticut, USA
Lecturer, Harvard Medical School (retired)
Boston, Massachusetts, USA

Geoffrey Hallmann, LLB, DipLegPrac
Former Lawyer (Disability and Compensation)
Lismore, New South Wales, Australia

Maureen Hanson, PhD
Liberty Hyde Bailey Professor
Department of Molecular Biology and Genetics
Cornell University
Ithaca, New York, USA

Malcolm Hooper, PhD, BPharm, MRIC, CChem
Emeritus Professor of Medicinal Chemistry
University of Sunderland
Tyne and Wear, England, UK

Leonard A. Jason, PhD
Professor of Psychology
DePaul University
Chicago, Illinois, USA

Michael W. Kahn, MD
Assistant Professor of Psychiatry
Harvard Medical School
Boston, Massachusetts, USA

Jon D. Kaiser, MD
Clinical Faculty
Department of Medicine
University of California, San Francisco
San Francisco, California, USA

David L. Kaufman, MD
Center for Complex Diseases
Mountain View, California
Member, The ME/CFS Collaborative Research Center at Stanford
Palo Alto, California, USA

Betsy Keller, PhD, FACSM
Professor, Department of Exercise & Sport Sciences
Ithaca College
Ithaca, New York, USA

Nancy Klimas MD
Director, Institute for Neuro-Immune Medicine
Nova Southeastern University
Director, Miami VA Medical Center GWI and CFS/ME Program
Miami, Florida, USA

Andreas M. Kogelnik, MD, PhD
Director
Open Medicine Institute
Mountain View, California, USA

Richard Kwiatek, MBBS, FRACP
Rheumatologist and Independent Researcher
Northern Adelaide Local Health Network
Adelaide, South Australia, Australia

Eliana M. Lacerda, MD, MSc, PhD
Clinical Assistant Professor
International Centre for Evidence in Disability
Faculty of Infectious and Tropical Diseases
London School of Hygiene & Tropical Medicine
London, England, UK

Charles W. Lapp, MD
Medical Director
Hunter-Hopkins Center
Charlotte, North Carolina, USA

Bruce Levin, PhD
Professor of Biostatistics
Columbia University
New York, New York, USA

Donald Lewis, MBBS, FRACGP, DRACOG
Medical Director
CFS Discovery
Melbourne, Victoria, Australia

Alan R. Light, PhD
Professor of Anesthesiology
Professor of Neurobiology and Anatomy
University of Utah
Salt Lake City, Utah, USA

Vincent C. Lombardi, PhD
Director of Research
Nevada Center for Biomedical Research
Reno, Nevada, USA

Alex Lubet, PhD
Professor of Music
Head, Interdisciplinary Graduate Group in Disability Studies
Affiliate Faculty, Center for Bioethics
Affiliate Faculty, Center for Cognitive Sciences
University of Minnesota
Minneapolis, Minnesota, USA

Steven Lubet, JD
Williams Memorial Professor of Law
Northwestern University Pritzker School of Law
Chicago, Illinois, USA

David F. Marks, PhD
Editor
Journal of Health Psychology
& Health Psychology Open
London, England, UK

Sonya Marshall-Gradisnik, PhD
Professor of Immunology
Co-Director, National Centre for Neuroimmunology and Emerging Diseases
Griffith University
Gold Coast, Queensland, Australia

Marlon Maus, MD, DrPH, FACS
DrPH Program Director
School of Public Health
University of California, Berkeley
Berkeley, California, USA

Neil R McGregor. BDS, MDSc, PhD
Clinical Associate Professor
Faculty of Medicine, Dentistry and Health Sciences
Bio21 Molecular Science & Biotechnology Institute
University of Melbourne.
Melbourne, Victoria, Australia

Patrick E. McKnight, PhD
Professor of Psychology
George Mason University
Fairfax, Virginia, USA

Marvin S. Medow, PhD
Professor of Pediatrics and Physiology
Chairman, New York Medical College IRB
Associate Director of The Center for Hypotension
New York Medical College
Hawthorne, New York, USA

Jose G. Montoya, MD, FACP, FIDSA
Professor of Medicine
Division of Infectious Diseases and Geographic Medicine
Stanford University School of Medicine
Stanford, California, USA
Director, Palo Alto Medical Foundation Toxoplasma Serology Laboratory
National Reference Center for the Study and Diagnosisof Toxoplasmosis
Palo Alto, California, USA

Sarah Myhill, MBBS
Clinician in Private Practice
Knighton, Wales, UK

Luis Nacul, MD, PhD
Clinical Associate Professor
International Centre for Evidence in Disability
Faculty of Infectious and Tropical Diseases
London School of Hygiene & Tropical Medicine
London, England, UK

Heidi Nicholl, PhD
Chief Executive Officer
Emerge Australia
Melbourne, Victoria, Australia

James M. Oleske, MD, MPH
François-Xavier Bagnoud Professor of Pediatrics
Senator of RBHS Research Centers, Bureaus, and Institutes
Director of Division of Pediatrics Allergy, Immunology & Infectious Diseases
Department of Pediatrics
Rutgers New Jersey Medical School
Newark, New Jersey, USA

Elisa Oltra, PhD
Professor of Molecular and Cellular Biology
Catholic University of Valencia School of Medicine
Valencia, Spain

Nigel Paneth, MD, MPH
University Distinguished Professor
Depts of Epidemiology & Biostatistics and Pediatrics & Human Development
College of Human Medicine
Michigan State University
East Lansing, Michigan, USA

Richard Podell, MD, MPH
Clinical Professor, Department of Family Medicine
Rutgers-Robert Wood Johnson Medical School
New Brunswick, New Jersey, USA

Nicole Porter, PhD
Psychologist in Private Practice
Rolling Ground, Wisconsin, USA

Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, New York, USA

Arthur L. Reingold, MD
Professor of Epidemiology
University of California, Berkeley
Berkeley, California, USA

Peter C. Rowe, MD
Professor of Pediatrics
Johns Hopkins University School of Medicine
Baltimore, Maryland, USA

Michael Scott, PhD
Clinician/Researcher
Psychological Therapies Unit
Liverpool, England, UK

Charles Shepherd, MB BS
Honorary Medical Adviser to the ME Association
Buckingham, England, UK

Christopher R. Snell, PhD
Scientific Director
WorkWell Foundation
Ripon, California, USA

Nigel Speight, MA, MB, BChir, FRCP, FRCPCH, DCH
Pediatrician
Durham, England, UK

Donald R. Staines, MBBS, MPH, FAFPHM, FAFOEM
Clinical Professor
Menzies Health Institute Queensland
Co-Director
National Centre for Neuroimmunology and Emerging Diseases
Griffith University
Gold Coast, Queensland, Australia

Philip B. Stark, PhD
Professor of Statistics
University of California, Berkeley
Berkeley, California, USA

Eleanor Stein, MD, FRCP(C)
Psychiatrist in Private Practice
Assistant Clinical Professor
University of Calgary
Calgary, Alberta, Canada

Staci Stevens, MA
Founder, Exercise Physiologist
Workwell Foundation
Ripon, California, USA

Julian Stewart, MD, PhD
Professor of Pediatrics, Physiology and Medicine
Associate Chairman for Patient Oriented Research
Director, Center for Hypotension
New York Medical College
Hawthorne, New York, USA

Leonie Sugarman, PhD
Emeritus Associate Professor of Applied Psychology
University of Cumbria
Carlisle, England, UK

John Swartzberg, MD
Clinical Professor Emeritus
School of Public Health
University of California, Berkeley
Berkeley, California, USA

Ronald G. Tompkins, MD, ScD
Summer M Redstone Professor of Surgery
Harvard Medical School
Boston, Massachusetts, USA

Barbara True, MD, FRACP
Private Practice
Wakefield Rheumatology
Adelaide, South Australia, Australia

Samuel Tucker, MD
Former Assistant Clinical Professor of Psychiatry
University of California, San Francisco
San Francisco, California, USA

David Tuller, DrPH
Lecturer in Public Health and Journalism
University of California, Berkeley
Berkeley, California, USA

Rosemary A. Underhill, MBBS, MRCOG, FRCSE
Physician, Independent Researcher
Palm Coast, Florida, USA

Derya Unutmaz, MD
Professor
The Jackson Laboratory for Genomic Medicine
Farmington, Connecticut, USA

AM Uyttersprot, MD
Neuropsychiatrist
AZ Jan Portaels
Vilvoorde, Belgium

Rosamund Vallings, MNZM, MBBS
General Practitioner
Auckland, New Zealand

Linda van Campen, MD
Cardiologist
Stichting Cardiozorg
Hoofddorp, The Netherlands

Mark VanNess, PhD
Professor of Health, Exercise & Sports Sciences
University of the Pacific
Stockton, California, USA
Workwell Foundation
Ripon, California, USA

Mark Vink, MD
Family Physician
Soerabaja Research Center
Amsterdam, The Netherlands

Frans Visser, MD
Cardiologist
Stichting Cardiozorg
Hoofddorp, The Netherlands

Tony Ward, MA (Hons), PhD, DipClinPsyc
Registered Clinical Psychologist
Professor of Clinical Psychology
School of Psychology
Victoria University of Wellington
Wellington, New Zealand
Adjunct Professor, School of Psychology
University of Birmingham
Birmingham, England, UK
Adjunct Professor, School of Psychology
University of Kent
Canterbury, England, UK

William Weir, FRCP
Infectious Disease Consultant
London, England, UK

John Whiting, MD
Specialist Physician in Private Practice
Brisbane, Queensland, Australia

Sadie Whittaker, PhD
Chief Scientific Officer
Solve ME/CFS Initiative
Los Angeles, California, USA

Carolyn Wilshire, PhD
Senior Lecturer
School of Psychology
Victoria University of Wellington
Wellington, New Zealand

Marcie Zinn, PhD
Cognitive Neuroscience and Data Science
Center for Community Research
DePaul University
Chicago, Illinois, USA
Associate Editor, BMC Journal of Translational Medicine

___________

[1] White PD et al. 2011. Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial. The Lancet, 377: 823–836

[2] Boseley S. 2011. Study finds therapy and exercise best for ME. The Guardian, 18 Feb. Available at: https://www.theguardian.com/society/2011/feb/18/study-exercise-therapy-me-treatment (accessed on April 23, 2018)

[3] Bleijenberg G, Knoop H. 2011. Chronic fatigue syndrome: where to PACE from here? The Lancet, 377: 786-788

[4] Wilshire C et al. 2016. Can patients with chronic fatigue syndrome really recover after graded exercise or cognitive behavioural therapy? A critical commentary and preliminary re-analysis of the PACE trial. Fatigue: Biomedicine, Health & Behavior, 14 Dec. Available at: http://www.tandfonline.com/doi/full/10.1080/21641846.2017.1259724 (accessed on April 23, 2018)

[5] U.S. Institute of Medicine (now National Academy of Medicine). 2015. Beyond myalgic encephalomyelitis/chronic fatigue syndrome: redefining an illness. The National Academies: Washington, DC, USA.

[6] Tuller D. 2015. Trial by error: the troubling case of the PACE chronic fatigue syndrome trial. VirologyBlog, 21-23 Oct. Available at: https://www.virology.ws/2015/10/21/trial-by-error-i/ (accessed onApril 23, 2018)

[7] Racaniello V. 2016. An open letter to The Lancet, again. VirologyBlog, 10 Feb. Available at: https://www.virology.ws/2016/02/10/open-letter-lancet-again/ (accessed on April 23, 2018)

[8] Rehmeyer J. 2016. Bad science misled millions with chronic fatigue syndrome. Here’s how we fought back. STAT, 21 Sept. Available at: https://www.statnews.com/2016/09/21/chronic-fatigue-syndrome-pace-trial/ (accessed on April 23, 2018)

[9] Geraghty K. 2017. ‘PACE-Gate’: when clinical trial evidence meets open data access. Journal of Health Psychology, 22: 1106-1112

[10] Wilshire C et al. 2018. Rethinking the treatment of chronic fatigue syndrome, a reanalysis and evaluation of findings from a recent major trial of graded exercise and CBT. BMC Psychology; published online 22 March. Available at: https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-018-0218-3 (accessed on April 23, 2018)

[11] Smith M et al. 2016. Diagnosis and treatment of myalgic encephalomyelitis/chronic fatigue syndrome; addendum. U.S. Agency for Healthcare Research and Quality. July. Available at: https://www.ncbi.nlm.nih.gov/books/NBK379582/ (accessed on April 23, 2018)

[12] Rehmeyer J, Tuller D. 2017. Why did it take the CDC so long to reverse course on debunked treatments for chronic fatigue syndrome? STAT, 25 Sept. Available at: https://www.statnews.com/2017/09/25/chronic-fatigue-syndrome-cdc/ (accessed on April 23, 2018)

[13] Whipple T. 2017. Mutiny by ME sufferers forces a climbdown on exercise treatment. The Times, 25 Sept.

[14] Health Council of the Netherlands. 2018. More scientific research on ME/CFS is needed to serve patients better. 19 March. Available at: https://www.gezondheidsraad.nl/en/news/more-scientific-research-on-mecfs-is-needed-to-serve-patients-better (accessed on April 23, 2018)

[15] Tucker M. 2018. Much can be done to ease ‘chronic fatigue syndrome’ symptoms. Medscape, 12 March. Available at: https://www.medscape.com/viewarticle/893766 (accessed on April 23, 2018)

(Many thanks to Mary Dimmock for helping to contact the signatories.)