The Validity of CFS Research Findings Under Fire

The question is, are CFS researchers manipulating results with selective reporting of their findings?

By PENNY SWIFT

The latest research report based on the widely criticized 2011 PACE Trial Study that recommends exercise for chronic fatigue syndrome (CFS) patients, is thwart with issues that compromise its credibility, according to a psychologist in New Zealand.

In addition the discovery by the ME/CFS community that at least one of its authors, Prof. Peter D. White has arrived at totally opposite conclusions in another CFS research study adds fuel to the fire.

Behavioral Intervention Studies Can be Harmful

Reacting to the controversial report, Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial, published online in The Lancet Psychiatry on January 13, an experimental psychologist from New Zealand has leveled sharp criticism stating that this type of behavioral intervention study “can do great harm in less direct ways” than “drug intervention studies.”

Dr. Carolyn E. Wilshire, a senior lecturer at Victoria University of Wellington’s School of Psychology in New Zealand compared the two types of studies and concluded that behavioral research should not have “a ‘get out of jail free’ card when it comes to scientific rigor.”

Five researchers, including Prof. White stated in The Lancet Psychiatry article that when it comes to CFS patients and exercise, “fear avoidance beliefs” help perpetuate both disability and ongoing “fatigue” in the illness. This has led to a widespread public outcry internationally by both patients and professionals with hands-on experience working with CFS sufferers.

Writing in BMJ online yesterday, Wilshire said drug interventions have an agreed set of quality standards, and research authors could not “selectively report only the most favorable outcome measures.” Behavioral interventions on the other hand are not consistently evaluated by these same standards, and this can lead to “exaggerated claims as to their effectiveness.”

Any assumption that rigorous standards are not required for behavioral interventions because the risk of harm is lower than that for drug interventions must be challenged in the case of ME/CFS research she says. This is because of the factual evidence that graded exercise therapy (GET) may result in adverse effects for patients.

She also criticized British mainstream media reports that supported a view of ME/CFS that minimized its severity, exaggerated patients’ responsiveness to treatment, and ultimately placed “responsibility for the illness back on the patient.”

The PACE Study

The PACE Trial study that Wilshire says (quoting references in her article) cost UK taxpayers £5-million, resulted in a report that she says fits the “behavioral interventions” model.

According to the original PACE study report, Comparison of adaptive pacing therapy, cognitive behavior therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomized trial, the condition of 61 percent of participants had improved a year after they had completed the 24-week GET program. This result was based on a combination of self-rated measures of physical function and fatigue. In other words the patients themselves assessed their improvement ratio. While this appeared “impressive” on the face of it, she points out that the specialist medical care “control condition” reported a similarly impressive 45 percent improvement.

This in itself casts doubt on the validity of the self-report measures used to assess improvement. Carolyn E. Wilshire

Because improvement on both sides was rated so high, “conclusions rest on the differences in improvement rates between the various conditions” – which amounted to only 14 to 16 percent of the people participating in the study.

This ties in with ME Research UK’s observation, reported in The Argus Report last week, that the original PACE Trial data shows only 10 to 15 percent of ME/CFS patients in the trials benefitted from GET.

Simple mathematics (based on their figures) shows that in reality, this can only represent at most, 102 people, out of the many millions known to have been diagnosed with the syndrome worldwide.

For this reason, says Wilshire, the design of no-treatment control is crucial.

Drug intervention studies include the use of a placebo that “controls spurious factors” that are known to affect outcomes, including patients’:

• expectation for improvement, and
• degree of investment in the study.

However, specialist medical care was used as the baseline condition in the PACE CFS research study, which, as Wilshire explains, was not an adequate control for either of these factors. It was unlikely that those in intervention groups and those with the condition would have “the same expectations of improvement.” They would also not have been likely to develop “the same kind of rapport with the therapist” or invest as much effort in the so-called treatment.

The PACE study also relied on self-report measures, which, says Wilshire, are considerably more vulnerable to the placebo effect that other more objective measures. Furthermore, treatment was “unblinded” and there was not adequate control of all the factors that influence “placebo responding.” To overcome this, it would have been essential to get objective outcomes from “blinded raters,” which didn’t happen.

The PACE CFS research study only reported one objective outcome, which was: “the average distance walked in six minutes increased after all treatments but reliably more so after GET.” While there were other “objective measures” that had been planned in the original protocol, these were never reported, she says.

All other positive outcomes are from self report. Carolyn E. Wilshire

Selective Reporting of Research Studies

Research is supposed to be objective, which is why there is major criticism of current psychological research standards that allow “the highly selective reporting of outcome measures.”

The problem is quite simple: our criterion for statistical significance (less than 5% probability of obtaining a significant effect by chance alone) means that up to 1 in every 20 statistical result could very well be artefactual. Carolyn E. Wilshire

What this implies is that researchers who measure “multiple outcomes” and then selectively report on only favorable outcomes, are manipulating the results; or as she puts it, “concealing from the reader this heightened probability of a spurious result.”

She goes on to argue that the researchers’ claim that the PACE CFS research study demonstrated “substantive evidence of genuine treatment effects” is dubious, especially since this is the study’s starting premise.

She is also highly critical of the new research report, published last week, four years after the original report. While it makes a “valuable contribution” by reporting a new objective measure in the form of the heart rate after a step exercise fitness test, the GET patients who were predicted to show the greatest improvement in fact did not differ from the non-treatment control. Furthermore, there was a distinct “lack of prominence” of this fitness test in the original paper.

This result leads to further concerns about the selective reporting in the study and the heavy reliance on self-report. Carolyn E. Wilshire

Ultimately, Wilshire says that if psychiatric and medical journals like The Lancet Psychiatry continue to publish behavioral studies, they need to be aware of recent psychology literature that discusses general weaknesses in psychological methodology.

This is important since the average newspaper writer clearly considers publications like The Lancet Psychiatry to be reliable sources of research material, and few will question what is presented.

Contradictory Research Results

Selective reporting is one thing, but what about contradictory research results?

A current topic of discussion in the ME/CFS community is the fact that the first listed author (of a total of 19) of the original PACE Trial report, who is also one of five authors responsible for the controversial PACE report published last week, Prof. Peter D. White, was also one of five authors of a previous study report that totally contradicts last week’s report. He is the only author who was involved with all three studies. None of his original co-authors were authors of the PACE Trial study. Three of his four co-authors in last week’s report were also co-authors of the first (of six) PACE Study reports.

The two contradictory studies are:

1. Is the chronic fatigue syndrome an exercise phobia? A case control study. April 2005
2. Rehabilitative therapies for chronic fatigue syndrome: a secondary mediation analysis of the PACE trial. January 2015

Study 1 set out to test whether patients with CFS have an exercise phobia by “measuring anxiety-related physiological and psychological reactions to ordinary activity and exercise.”

The data suggest that CFS patients without a comorbid psychiatric disorder do not have an exercise phobia.

Study 2 set out to show that cognitive behavior therapy (CBT) added to specialist medical care (SMC), or GET added to SMC, is more effective in reducing fatigue and improving physical function that both adaptive pacing therapy (APT) plus SMC and SMC alone for CFS.

Our main finding was that fear avoidance beliefs were the strongest mediator for both CBT and GET. Changes in both beliefs and behavior mediated the effects of both CBT and GET, but more so for GET. The results support a treatment model in which both beliefs and behavior play a part in perpetuating fatigue and disability in chronic fatigue syndrome.

This translates directly to patients having exercise phobia.

While the PACE Trial report was published in 2011, the trials took place between March 2005 and November 2008. This means that they began about a month after the paper White co-authored, that found CFS patients did not have an exercise phobia, was published.

Study 1 was published by the US National Institutes of Health (NIH) National Library of Medicine, PubMed.

The PACE Trial report was published in The Lancet Psychiatry in March, 2011, and republished (with acknowledgement to The Lancet) by the NIH PubMed library.

Study 2 was published in The Lancet Psychiatry. PubMed Health carried a story about the report titled How therapy and exercise ‘may help some with CFS’. This is based on British newspaper reports, including those published by The Guardian, the Daily Mail, The Independent and The Daily Telegraph, all of which are given as references along with a link to the original article published in The Lancet Psychiatry.

Prof. Peter D. White

Prof. White is, according to a bibliography published by the UK Government, Professor of Psychological Medicine at Barts and the London Medical School, Queen Mary University of London. He is also a “consultant liaison psychiatrist” at St. Bartholomew’s Hospital where he is a co-leader of the CFS service. He assesses and cares for patients with both physical and mental health problems (cancer, depression and CFS are specified).

In addition, he has acted as an advisor to government departments in the UK and to a re-insurance company about “specific and general disability related to both functional somatic ill health and mental ill health.”

The biography also states that his research “has helped to establish the independent existence of CFS” as well as its “relationship with other functional somatic syndromes,” and “the place of infections and exercise intolerance in causing and maintaining CFS.”

The two CFS research studies cited in this article are only two of many Prof. White has authored or co-authored.