What is the aim of this review?
The aim of this Cochrane Review was to find out if external inspection of compliance with standards can improve improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes. Cochrane researchers collected and analysed all relevant studies to answer this question and found two studies.
Key messages
It is unclear whether third-party inspection programmes designed to measure a healthcare organisation's compliance with standards of care can improve professional practice and healthcare outcomes. There was little information on patient outcomes. This review highlights the lack of high-quality evaluations.
What was studied in the review?
Third-party (external) inspection programmes are used within healthcare settings (e.g. clinics and hospitals) to increase the compliance with evidence-based standards of care, but very little is known of their benefits in terms of organisational performance (e.g.,waiting list times, inpatient length of stay),the performance of healthcare professionals (e.g. referral rate, prescribing rate) and patient outcomes (e.g. mortality, and condition specific outcomes like blood glucose for diabetic patients, or weight loss for overweight or obese patients), or their cost-effectiveness. Accreditation programmes are one example of an external review system. Accreditation is a process of review that healthcare organisations participate in to demonstrate the ability to meet predetermined criteria and standards of accreditation established by a professional accrediting agency. Infection control inspection programmes are also examples. Designed to reduce infection, standards based on evidence and best practice, are used to improve care quality and safety to decrease healthcare-acquired infection rates (also called a hospital acquired infection or nosocomial infection, is an infection that is acquired at a hospital or another healthcare setting). If the standards are not adhered to, the external inspection body can take actions to reinforce them.
What are the main results of the review?
The review authors searched the literature for studies evaluating the effects of external inspection of compliance with standards and found two relevant studies: one study involving 20 hospitals in the Republic of South Africa and one study providing time series data (a sequence of outcome measurements taken at successive equally spaced points in time) involving all acute hospital trusts in England (a trust is a National Health Service hospital which has opted to withdraw from local authority control and be managed by a trust instead). The comparison was no inspection.
One study reported improved compliance scores with hospital accreditation standards. However, it is uncertain whether external inspection leads to improved compliance with standards because the certainty of the evidence was very low. Only one of the nine intervention hospitals achieved accreditation status during the study period.
Another study reported on the effects of an infection control inspection programme. This programme was commissioned in the UK to reduce infection rates of MRSA (methicillin-resistant Staphylococcus aureus, which is a form of bacterial infection that is resistant to many antibiotics). However, the inspection programme was only one element of a wider range of methods being applied to infection control in the UK National Health Service at that time. Even before the introduction of the inspection programme, the infection rates appeared to be decreasing - but the introduction of the inspection programme did not accelerate this decrease. It is also uncertain whether the Healthcare Commission's Healthcare Associated Infections Inspection Programme may lead to lower MRSA infection rates or not because the certainty of the evidence was very low.
How up-to-date is this review?
The review authors searched for studies that had been published up to June 2015.
The review highlights the paucity of high-quality controlled evaluations of the effectiveness and the cost-effectiveness of external inspection systems. If policy makers wish to understand the effectiveness of this type of intervention better, there needs to be further studies across a range of settings and contexts and studies reporting outcomes important to patients.
Inspection systems are used in healthcare to promote quality improvements (i.e. to achieve changes in organisational structures or processes, healthcare provider behaviour and patient outcomes). These systems are based on the assumption that externally promoted adherence to evidence-based standards (through inspection/assessment) will result in higher quality of healthcare. However, the benefits of external inspection in terms of organisational-, provider- and patient-level outcomes are not clear. This is the first update of the original Cochrane review, published in 2011.
To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes.
We searched the following electronic databases for studies up to 1 June 2015: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, Database of Abstracts of Reviews of Effectiveness, HMIC, ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform. There was no language restriction and we included studies regardless of publication status. We also searched the reference lists of included studies and contacted authors of relevant papers, accreditation bodies and the International Organization for Standardization (ISO), regarding any further published or unpublished work. We also searched an online database of systematic reviews (PDQ-evidence.org).
We included randomised controlled trials (RCTs), non-randomised trials (NRCTs), interrupted time series (ITSs) and controlled before-after studies (CBAs) evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes in hospitals, primary healthcare organisations and other community-based healthcare organisations.
Two review authors independently applied eligibility criteria, extracted data and assessed the risk of bias of each included study. Since meta-analysis was not possible, we produced a narrative results summary. We used the GRADE tool to assess the certainty of the evidence.
We did not identify any new eligible studies in this update. One cluster RCT involving 20 South African public hospitals and one ITS involving all acute hospital trusts in England, met the inclusion criteria. A trust is a National Health Service hospital which has opted to withdraw from local authority control and be managed by a trust instead.
The cluster RCT reported mixed effects of external inspection on compliance with COHSASA (Council for Health Services Accreditation for South Africa) accreditation standards and eight indicators of hospital quality. Improved total compliance score with COHSASA accreditation standards was reported for 21/28 service elements: mean intervention effect was 30% (95% confidence interval (CI) 23% to 37%) (P < 0.001). The score increased from 48% to 78% in intervention hospitals, while remaining the same in control hospitals (43%). The median intervention effect for the indicators of hospital quality of care was 2.4% (range -1.9% to +11.8%).
The ITS study evaluated compliance with policies to address healthcare-acquired infections and reported a mean reduction in MRSA (methicillin-resistant Staphylococcus aureus) infection rates of 100 cases per quarter (95% CI -221.0 to 21.5, P = 0.096) at three months' follow-up and an increase of 70 cases per quarter (95% CI -250.5 to 391.0; P = 0.632) at 24 months' follow-up. Regression analysis showed similar MRSA rates before and after the external inspection (difference in slope 24.27, 95% CI -10.4 to 58.9; P = 0.147).
Neither included study reported data on unanticipated/adverse consequences or economic outcomes. The cluster RCT reported mainly outcomes related to healthcare organisation change, and no patient reported outcomes other than patient satisfaction.
The certainty of the included evidence from both studies was very low. It is uncertain whether external inspection accreditation programmes lead to improved compliance with accreditation standards. It is also uncertain if external inspection infection programmes lead to improved compliance with standards, and if this in turn influences healthcare-acquired MRSA infection rates.