The Cochrane Review 'Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality' has recently published. We talked to author Camilla Hansen to learn what the latest evidence says and why it is important.
Can you please tell us about this Cochrane Review?
"Systematic reviews summarise existing evidence on a specific research question. These summaries sometimes provide a basis for developing clinical guidelines and can have a major impact on how patients are treated. It is therefore essential that findings from systematic reviews are trustworthy. If not, we may end up wasting resources on interventions that don’t actually work, or even harming patients by using interventions that are dangerous, rather than helpful.
There is a lot of debate about the impact of financial conflicts of interest on research findings and their interpretation. This exact issue has been investigated in many methodology studies of financial conflicts of interest in primary research studies, mainly clinical trials; and these studies have shown that financial conflicts of interest do impact on the conclusions drawn from the randomised trials.
In contrast, fewer methodological studies have investigated the impact of financial conflicts of interest in systematic reviews, and we are not aware of any previous attempt to bring these studies together into a methodological systematic review. We decided to fill this gap.
We included ten studies that investigated a total of 1010 systematic reviews. We primarily investigated the potential impact on results (divided into estimated treatment effects and statistically favourable results) and conclusions. We found no clear difference in results between systematic reviews with and without financial conflicts of interest, but our findings are based on very limited data with a high degree of uncertainty. However, the evidence is stronger in relation to the conclusions of systematic reviews and we found that those with financial conflicts of interest more often had favourable conclusions than systematic reviews without financial conflicts of interest. This is based on seven methodology studies that looked at a total of 411 systematic reviews.
We also investigated the potential impact on the quality of the systematic reviews and found a tendency that systematic reviews with financial conflicts of interest had lower methodological quality than those without financial conflicts of interest.
What are financial conflicts of interest in this context and why are they important?
Financial conflicts of interest can occur when either a company or individuals may benefit financially from how the evidence in a systematic review is interpreted and used, and our focus was on those that relate to connections with drug and device companies. The issue can arise when a systematic review is directly funded by such a company or done by authors employed by such companies. It can also happen when a systematic review is conducted by authors who have financial relationships with drug or device companies, for example when authors have received fees or research grants, or are on advisory boards of drug and device companies. The Cochrane review did not investigate systematic reviews done by authors with a personal financial and professional interest related to certain interventions. For example, surgeons doing specific fee-based procedures in private practice. Similarly, the Cochrane review did not investigate academic conflicts of interest where researchers’ grant applications and academic careers may be influenced by positive studies published in high impact journals.
How did this review come about?
I am doing a PhD on conflicts of interest in clinical research. I realised, together with my supervisors, that no methodological systematic review had been published on this important topic. We therefore decided to do a Cochrane Methodology Review.
What sort of funding can present a conflict of interest? And why?
In medical research, funding can present conflicts of interest in a variety of ways. Sometimes, systematic reviews are primarily funded by non-profit sources (e.g. when reviews are conducted by academic researchers employed at public universities and financed through their salaries), but occasionally industry provides additional indirect funding by contributing with assistance from medical writer or company statisticians. In some cases, systematic reviews are funded by non-profit sources, but conducted by authors with financial ties to the manufacturer of the drug or device they are reviewing. At other times, systematic reviews are solely financed through industry funding (e.g. when drug or device companies provide funding and salary for academic researchers at public institutions to conduct a specific review).
Common for all types of funding is that it can create ties between the researchers and industry. In some cases, the funding company may influence the design, analysis, or reporting of the study. Since the company has an interest in reaching a specific conclusion, this can impact on decisions made in the way that a study is carried out. In other cases, the conflicts of interest may result in the academic researchers being unduly influenced by the interests of the funder, when interpreting the evidence.
For this review, we focused on funding from drug and device companies only. However, previous studies have also demonstrated that funding from other industries, e.g. tobacco and nutrition companies, impact on research findings and interpretation. Recently, it has been debated whether funding from non-profit organisations or governmental institutions may also create conflicts of interest since they may have an interest in a particular direction of the findings of a study. However, currently little research has been done on the topic.
How does your review build on the work of the previously published review update in 2017 that concluded “sponsorship of drug and device studies by manufacturing company leads to more favourable efficacy results and conclusions than sponsorship by other sources?”
The review from 2017 was led by my colleague Andreas Lundh and focused on funding in primary research studies, mainly clinical trials. Although we deployed similar methods in our review, our focus was on financial conflicts of interest in systematic reviews, rather than the trials in those reviews.
We are also working on a third Cochrane Methodology Review focusing on conflicts of interest in a group of non-research publications, namely clinical guidelines, transcripts from drug and device advisory committee meetings, opinion pieces such as editorials, and narrative reviews.
Therefore, there will soon be three Cochrane Methodology Reviews investigating how conflicts of interest impact on three core types of publications that all influence patient care.
I was interested to read that in contrast to the 2017 review on primary studies, you did not find an association between financial interests and statistically favourable results in systematic reviews. Were you surprised?
You are right that we did not find such an association, but we only found two studies investigating the results sections of systematic reviews, which means that there is not enough data at the moment to determine whether or not there is an association.
You mentioned that you are working on another review: “Conflicts of interest in clinical guidelines, opinion pieces, and narrative reviews”. When is it due? Any preliminary findings you can share?
We are finalising that review right now, and it looks as though the findings from the 2017 review on clinical trials, the new one on systematic reviews, and that third Cochrane Review are quite similar.
"What can people who read and use systematic review evidence learn from your review?
Our findings suggest that systematic reviews with financial conflicts of interest should be interpreted carefully. People who read and use systematic review evidence should primarily look for systematic reviews without financial conflicts of interest. If that is not available, we suggest that users read the review conclusions with scepticism, are aware of the declarations of interest for the review authors and funding sources, and pay particular attention to their critical appraisal of the review’s methods."