Driving quality improvement through the NPCA Outlier Process

Julie Nossiter, NPCA Audit Lead

The NPCA’s aim is to stimulate and support local and national quality improvement interventions and – through annual re-auditing – assess the impact of these interventions. A key component is the ‘outlier process’, whereby the NPCA identifies where improvement is required.

Appropriate performance measures are essential when trying to understand the quality of care received by patients and how this varies by hospital. Many men experience treatment-related side effects after radical surgery or radiotherapy for localised prostate cancer including detrimental impacts on urinary, bowel and/or sexual functioning. To help us to understand the impact of these side-effects after radical treatment, the NPCA has developed ‘treatment-outcome’ measures that are meaningful and relevant to patients and clinicians .

For the first time in 2019, the NPCA publicly reported validated and precise measures detecting urinary complications following surgery (radical prostatectomy) and bowel complications following radiotherapy (external beam radiation [EBRT]). These were based on data from routine clinical datasets, independent of individual clinician reporting. We also published measures of urinary, bowel and sexual function reported by patients in the NPCA Patient Survey. These indicators are adjusted for differences in patients’ age, prostate cancer risk group, other underlying conditions and deprivation status, enabling a fair comparison of the outcomes among NHS Hospitals.

Using funnel plots to compare individual hospital results with the national average, we identify ‘potential negative outliers’ whose performance is outside acceptable limits (further from the national average than would usually occur by chance alone). This information is fed back to clinical teams providing an opportunity for the hospitals to review the completeness and accuracy of their data with support from the NPCA team. After a period of reflection and exploration of possible causes, clinical teams develop a local improvement plan which is published by the NPCA alongside the results. The NPCA team also provide these results to other national quality improvement endeavours such as the National Clinical Audit Benchmarking initiative which supports Care Quality Commission hospital inspections, NHS England’s Clinical Outcomes Publication programme and the Getting it Right First Time (GIRFT) programme.

A potential limitation of the outlier process is that by targeting only the limited number of ‘outlier’ hospitals we mirror a ‘high-risk’ approach (rather than a population approach) of preventing poor quality care, with improvement endeavours restricted to the hospitals affected. The NPCA Quality Improvement workshop provided an opportunity to ‘close the circle’ by bringing together clinicians from surgical and radiotherapy centres across England and Wales including poorer performers, performance exemplars (positive outliers whose outcomes are consistently better than the national average) and all hospitals in between (whose performance is within expected limits).

Clinicians from the ‘better’ performing centres illustrated the steps they’d taken, and the protocols they’d developed, to reduce treatment-related toxicity after radical treatment for prostate cancer. Clinicians from the ‘under’ performing centres bravely shared their experience of the outlier process, the efforts undertaken to evaluate their practices and processes of care, and the changes made as a result.

It is clear that the outlier process can act as a catalyst to stimulate and promote local quality improvement. However, in order to spread quality improvement that can be scaled up to a national level (the population approach), the NPCA outlier process is embedded within a reporting programme that disseminates the results for all hospitals and illustrates both examples of good practice and learnings from hospitals who are embarking upon an improvement journey. This enables trusts to understand their performance relative to their peers and to make improvements as required.

It was encouraging to see the level of engagement by clinicians across the board during the workshop and their commitment to improving the outcomes for men undergoing radical treatment. We are committed to reporting treatment outcomes each year and assessing whether the changes implemented translate into a reduction in complication rates. Future NPCA workshops will provide more opportunities to reflect on the lessons learnt, to share best practice and to spread improvement mechanisms.