Patient treatment practices are often based on clinical research. Systematic reviews are a core type of such clinical research. When several similar studies (i.e. studies investigating the same research questions using similar methods) have been conducted, these can be identified and analysed in a systematic review. Systematic reviews thereby summarise existing studies and provide an overview of a specific research field. Thus, systematic reviews may have a major influence on decisions about patient care and it is essential that such reviews are trustworthy.
Sometimes, systematic reviews are funded by companies with a financial interest in the review's results and conclusions, for example because they produce a drug or device investigated in the review. At other times, systematic reviews are carried out by researchers with a personal financial interest in a specific result, for example when the researcher acts as a consultant for the company producing an intervention that is assessed in the review. These financial conflicts of interest may impact on how systematic reviews are conducted and reported. Our Cochrane Methodology Review focuses on financial conflicts of interest related to drug or device companies in systematic reviews. Our primary aim was to investigate the degree to which systematic reviews with financial conflicts of interest present review results and make conclusions that are more favourable than systematic reviews without such financial conflicts of interest. Our secondary aim was to investigate the degree to which systematic reviews with financial conflicts of interest differ in methodological quality from systematic reviews without such financial conflicts of interest.
We found 10 studies comparing systematic reviews with and without financial conflicts of interest. Based on two of these studies, we found no evidence of a difference in review results between systematic reviews with and without financial conflicts of interest. Based on seven studies, we found that systematic reviews with financial conflicts of interest more often had conclusions favourable towards the experimental intervention (risk ratio (RR): 1.98, 95% confidence interval (CI): 1.26 to 3.11). Also, based on four studies, systematic reviews with financial conflicts of interest tended to have lower methodological quality (RR for 11 dimensions of methodological quality spanned from 1.00 to 1.83).
Our analyses suggest that when systematic reviews have financial conflicts of interest related to drug or device companies, they are of lower methodological quality, and have more favourable conclusions. However, it is not clear whether this derives from actual differences in the review's results or the over-interpretation of those results. Based on our findings, we suggest that people who use systematic reviews, including patients, clinicians, developers of clinical guidelines, and planners of future research, could primarily use systematic reviews without financial conflicts of interest. If such reviews are not available, we suggest that users are especially cautious when they read and interpret systematic reviews with financial conflicts of interest.
Systematic reviews with financial conflicts of interest more often have favourable conclusions and tend to have lower methodological quality than systematic reviews without financial conflicts of interest. However, it is uncertain whether financial conflicts of interest are associated with the results of systematic reviews. We suggest that patients, clinicians, developers of clinical guidelines, and planners of further research could primarily use systematic reviews without financial conflicts of interest. If only systematic reviews with financial conflicts of interest are available, we suggest that users read the review conclusions with skepticism, critically appraise the methods applied, and interpret the review results with caution.
Financial conflicts of interest in systematic reviews (e.g. funding by drug or device companies or authors' collaboration with such companies) may impact on how the reviews are conducted and reported.
To investigate the degree to which financial conflicts of interest related to drug and device companies are associated with results, conclusions, and methodological quality of systematic reviews.
We searched PubMed, Embase, and the Cochrane Methodology Register for studies published up to November 2016. We also read reference lists of included studies, searched grey literature sources, and Web of Science for studies citing the included studies.
Eligible studies were studies that compared systematic reviews with and without financial conflicts of interest in order to investigate differences in results (estimated treatment effect and frequency of statistically favourable results), frequency of favourable conclusions, or measures of methodological quality of the review (e.g. as evaluated on the Oxman and Guyatt index).
Two review authors independently determined the eligibility of studies, extracted data, and assessed risk of bias. We synthesised the results of each study relevant to each of our outcomes. For meta-analyses, we used Mantel-Haenszel random-effects models to estimate risk ratios (RR) with 95% confidence intervals (CIs), with RR > 1 indicating that systematic reviews with financial conflicts of interest more frequently had statistically favourable results or favourable conclusions, and had lower methodological quality. When a quantitative synthesis was considered not meaningful, results from individual studies were summarised qualitatively.
Ten studies with a total of 995 systematic reviews of drug studies and 15 systematic reviews of device studies were included. We assessed two studies as low risk of bias and eight as high risk, primarily because of risk of confounding. The estimated treatment effect was not statistically significantly different for systematic reviews with and without financial conflicts of interest (Z-score: 0.46, P value: 0.64; based on one study of 14 systematic reviews which had a matched design, comparing otherwise similar systematic reviews). We found no statistically significant difference in frequency of statistically favourable results for systematic reviews with and without financial conflicts of interest (RR: 0.84, 95% CI: 0.62 to 1.14; based on one study of 124 systematic reviews). An analysis adjusting for confounding due to methodological quality (i.e. score on the Oxman and Guyatt index) provided a similar result. Systematic reviews with financial conflicts of interest more often had favourable conclusions compared with systematic reviews without (RR: 1.98, 95% CI: 1.26 to 3.11; based on seven studies of 411 systematic reviews). Similar results were found in two studies with a matched design, which therefore had a reduced risk of confounding. Systematic reviews with financial conflicts of interest tended to have lower methodological quality compared with systematic reviews without financial conflicts of interest (RR for 11 dimensions of methodological quality spanned from 1.00 to 1.83). Similar results were found in analyses based on two studies with matched designs.