What is the aim of this review?
Relapse and recurrence (becoming unwell again after making an improvement) are common in depression and lead to increased disability and decreased quality of life for patients. Relapse is a re-emergence of the initial episode of depression after some initial improvement, whereas recurrence is the onset of a new episode of depression after recovery. Outcomes, such as relapse and recurrence, can sometimes be predicted while people are well, using information available at the time. A mathematical calculation can be performed to assess an individual person's risk; this calculation is known as a 'prognostic model' or a prediction tool. In most health services, including the National Health Service (NHS) in the UK, resources such as doctors and therapists need to be used in the best way possible, for the people who will gain the most benefit from them. If accurate prediction tools are available, the information can be used to identify the most 'high risk' patients and make sure they receive additional support to try to prevent a relapse or a recurrence.
The aim of this review was to identify studies that have attempted to develop a prediction tool for relapse or recurrence of depression in adults. We were interested in studies that had attempted to make this prediction while patients were well. We also included tools that predicted the chance of patients staying well. If we had found multiple studies that tested the same prediction tool, we planned to combine these to work out a better summary of how well that tool worked.
Key messages
We identified 10 prediction tools (over 11 studies) for relapse or recurrence. These were either not proven to be good at predicting relapse/recurrence, or the studies had problems with how they were carried out, meaning that none of the prediction tools were at a stage where they could be used in the real world. Further work is needed to improve prediction of relapse or recurrence of depression.
What was studied in the review?
We collected and analysed the results of 11 relevant studies. We were interested in several things: how researchers had defined relapse and recurrence (for example, whether they had used clinical interviews or self-report questionnaires to diagnose depressive symptoms); what information was gathered to help make predictions; the techniques used by the researchers to help develop the tools; and how well the tools predicted. We were also interested in whether the tools were tested in a separate group of participants, which is essential to ensure that the model can predict accurately in patients in the real world.
Finally, we assessed the studies to determine how confident we could be in the results, given the approaches taken by researchers (this is called 'risk of bias') and how relevant the studies were to our review (this is called 'applicability').
What are the main results of the review?
We found 11 studies. Ten of these developed different models and one study tested one of the models developed in a previous study. It was not possible to combine results for any particular tool.
Ten of the 11 studies were rated at high risk of bias. This means that we cannot be confident in the results that were presented, due to some issues with the way the studies were conducted. The most common issue was that there were not enough participants included in the studies. Other common problems involved the statistical approaches used by the researchers.
One study was at low overall risk of bias, which means that we can be more confident in trusting the results. However, this tool did not make accurate predictions about relapse or recurrence.
We found no studies that could be used in clinical practice; further work is needed to develop tools for predicting relapse or recurrence of depression.
How up-to-date is the review?
The literature search for this review was completed in May 2020.
Of the 10 prognostic models identified (across 11 studies), only four underwent external validation. Most of the studies (n = 10) were assessed as being at high overall risk of bias, and the one study that was at low risk of bias presented a model with poor predictive performance. There is a need for improved prognostic research in this clinical area, with future studies conforming to current best practice recommendations for prognostic model development/validation and reporting findings in line with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement.
Relapse (the re-emergence of depressive symptoms after some level of improvement but preceding recovery) and recurrence (onset of a new depressive episode after recovery) are common in depression, lead to worse outcomes and quality of life for patients and exert a high economic cost on society. Outcomes can be predicted by using multivariable prognostic models, which use information about several predictors to produce an individualised risk estimate. The ability to accurately predict relapse or recurrence while patients are well (in remission) would allow the identification of high-risk individuals and may improve overall treatment outcomes for patients by enabling more efficient allocation of interventions to prevent relapse and recurrence.
To summarise the predictive performance of prognostic models developed to predict the risk of relapse, recurrence, sustained remission or recovery in adults with major depressive disorder who meet criteria for remission or recovery.
We searched the Cochrane Library (current issue); Ovid MEDLINE (1946 onwards); Ovid Embase (1980 onwards); Ovid PsycINFO (1806 onwards); and Web of Science (1900 onwards) up to May 2020. We also searched sources of grey literature, screened the reference lists of included studies and performed a forward citation search. There were no restrictions applied to the searches by date, language or publication status .
We included development and external validation (testing model performance in data separate from the development data) studies of any multivariable prognostic models (including two or more predictors) to predict relapse, recurrence, sustained remission, or recovery in adults (aged 18 years and over) with remitted depression, in any clinical setting. We included all study designs and accepted all definitions of relapse, recurrence and other related outcomes. We did not specify a comparator prognostic model.
Two review authors independently screened references; extracted data (using a template based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS)); and assessed risks of bias of included studies (using the Prediction model Risk Of Bias ASsessment Tool (PROBAST)). We referred any disagreements to a third independent review author. Where we found sufficient (10 or more) external validation studies of an individual model, we planned to perform a meta-analysis of its predictive performance, specifically with respect to its calibration (how well the predicted probabilities match the observed proportions of individuals that experience the outcome) and discrimination (the ability of the model to differentiate between those with and without the outcome). Recommendations could not be qualified using the GRADE system, as guidance is not yet available for prognostic model reviews.
We identified 11 eligible prognostic model studies (10 unique prognostic models). Seven were model development studies; three were model development and external validation studies; and one was an external validation-only study. Multiple estimates of performance measures were not available for any of the models and, meta-analysis was therefore not possible. Ten out of the 11 included studies were assessed as being at high overall risk of bias. Common weaknesses included insufficient sample size, inappropriate handling of missing data and lack of information about discrimination and calibration. One paper (Klein 2018) was at low overall risk of bias and presented a prognostic model including the following predictors: number of previous depressive episodes, residual depressive symptoms and severity of the last depressive episode. The external predictive performance of this model was poor (C-statistic 0.59; calibration slope 0.56; confidence intervals not reported). None of the identified studies examined the clinical utility (net benefit) of the developed model.