In this interview with Jill Hayden we learn how this review, one of Cochrane’s largest reviews published to date, has led to new ways of working collaboratively and with an increased focus on research integrity. An ongoing update of this review will bring five related Cochrane reviews together into one.
 
You have just completed the Cochrane review of Exercise treatment for chronic low back pain. How is this Cochrane review different to others you’ve worked on?
The size of this review made it quite different from other systematic reviews that I have contributed to, with 249 included trials. The review was challenging and took several years to complete because of the number of trials. As with most teams conducting systematic reviews, we had limited funding available to support this research, so it was difficult to complete the review with numerous new eligible trials being published on the topic each year.

However, a benefit of a review this size is the ability to explore study features – we were able to conduct analyses and identify issues that would not have been possible in smaller, focused reviews.

The review also allowed us to reflect on issues of research waste, duplication of effort and more efficient evidence production. For example, we observed several smaller systematic reviews published on the same topic while we were working hard assessing and synthesizing the exact same trials. With these challenges and observations, we have proposed a new collaborative model of review. We are excited to have recently received funding from the Canadian Institutes of Health Research to pilot an update of the review using this approach, in collaboration with Cochrane. We are calling the model a ‘Network Systematic Review’.

Tell me more about the Network Systematic review model idea.
Our Cochrane review investigates the effectiveness of exercise treatments for chronic low back pain – a broad question that includes any type of exercise. We realized that there are several other Cochrane review teams who are conducting reviews of specific exercise types that fall directly under the broad ‘exercise review’. Plus, many overlapping reviews are published outside of Cochrane. Every team has been searching, screening, and extracting data from some of the exact same trials. Replication in science is important, but this seemed excessive… This duplication results in a lot of wasted time and energy in the low back pain evidence ecosystem.

With the network systematic review, we will work collaboratively on each of these review steps. Instead of small teams working in isolation, we will coordinate more than twenty international researchers to plan and conduct the reviews together.

What do you think this new proposed model could mean for Cochrane and how reviews are produced?
Cochrane infrastructure is perfectly arranged to facilitate this network systematic review approach; we feel very fortunate to be partnering with the Cochrane Musculoskeletal group on this initiative.

There are benefits of the collaborative review conduct (for example, multiple review teams challenging each other and reaching consensus on methods and interpretation), and also there are benefits of the outputs. The overarching review provides the ability to better describe and assess studies in the field, and also to conduct analyses such as comparing effectiveness of treatments with network meta-analyses. The more in-depth, focused sub-review topic investigations will provide evidence about clinically relevant questions.

We anticipate that this model will make it more feasible to keep evidence up to date, standardize review methods, and allow methods investigations, all while potentially streamlining editorial processes.

My vision, I know shared by others, will be that data from systematic reviews and the trials included in systematic reviews are accessible and maintained in a searchable format so that reviewers and guideline developers can build on past work and are not duplicating efforts unnecessarily with overlapping syntheses.

Within this review, you noticed duplicated data in two trials that led you to assess research integrity issues in a comprehensive way. How did you decide what trials to exclude from the review?
This is a great example of a benefit of our large Cochrane review. Due to the large number of included studies, we added a step in the review process to identify potential linked publications (multiple publications of the same trial) and duplicate data. During this check we identified the exact same results data reported in two otherwise unrelated studies, and we noticed apparent patterns of reporting in these trial reports and several other publications. These observations prompted our comprehensive investigation of trial research integrity and publication characteristics.

We took a step back and defined characteristics of trials that we thought may be related to the trustworthiness of the studies such as trial pre-registration, risk of bias, inadequate reporting, plagiarism, and presumed predatory publication.

In the end, our research integrity check led us to exclude 30 trials from the review, each of which we judged to have multiple integrity or publication concerns. We have described our detailed methods and these research integrity characteristics for all trials included in the review in a separate publication.

I am pleased to see Cochrane’s new problematic studies policy that addresses this issue. It is important that we better understand how to measure important integrity characteristics and use this to limit the impact of untrustworthy data in evidence syntheses.

• Dr Jill Hayden is lead author of this study
• Read the full review here
• Read the review in Malay
• Visit Cochrane Back and Neck Group