Dec 26, 2019, 10:05 PM by Yiou Huang
Logibec is in partnership with three healthcare organizations in Quebec, Canada, to develop and validate a readmission risk prediction model that will identify a patient’s likelihood of being readmitted within 45 days of discharge.
Known for being costly, unplanned readmissions are one of the greatest challenges for healthcare organizations who are working towards offering the best care treatments to their patients. Several discharge and transitional care interventions aimed at lowering the number of unplanned readmissions have emerged in various jurisdictions; however, there is limited evidence in literature to support an understanding of which interventions are most effective.
In addition, it is important to underline that the return on investment of these interventions depends greatly on the hospital’s ability to accurately assess which patients are at high risk of readmission and will most likely benefit from these resource-intensive discharge and transitional care interventions.
Readmission risk prediction models designed for healthcare organizations have gained momentum in recent years. This trend is due to a combination of better access to individual-level electronic data and improvements in computing power, which has made available the use of predictive models and tools to hospitals for preventive management of high risk readmission patients.
It is especially true in the United States since the introduction of the Medicare’s Hospital Readmissions Reduction Program (HRRP) under the Affordable Care Act that imposes a financial penalty on hospitals that have an excessive readmission rate.
Logibec’s Research Collaborative from Phase I to Phase II
With our partners – the Centre Hospitalier de l’Université de Montréal (CHUM), the North of Montreal Island Integrated University Health and Social Services Centre (CIUSSS du Nord de l’île de Montréal) and the Integrated Health and Social Services Centres of Laval (CISSS de Laval) – the two first objectives of the Research Collaborative consisted in reviewing the literature on existing readmission prediction models and in performing an external validation of the performance of the LACE index. Our initial results were presented at the 2017 International Health Economics Association Congress in Boston.
The LACE index uses four variables to predict the risk of death or unplanned readmission within 30 days after hospital discharge among medical and surgical patients. It was developed in Ontario in 2010 by van Walraven et al. using data from 11 hospitals.
By applying the LACE index on our partners’ historical data (all adult inpatients discharged with the exception of psychiatric, obstetric or palliative care admissions), we validated the performance of the LACE index in adequately predicting 30-day death or urgent readmission after hospital discharge.
Our results suggested that the LACE index is a poor predictor of death or readmission within 30 days for patients who were at their first admission (i.e. having no historical acute care data) with a concordance C-statistic below 0.7 in all three studies.
The C-statistic, one of many measures used to assess the performance of a prediction model, tells how well the model distinguishes patients who are readmitted within 30 days from those who are not readmitted. A C-statistic of 0.5 indicates that the model is no better than chance for making a prediction. Prediction models are typically considered reasonable when the discriminative ability (C-statistic) is higher than 0.7 and strong when the C-statistic exceeds 0.8.
Considering patients with multiple admissions by randomly selecting one of their episodes, the C-statistic of the LACE index improved to 0.72 for two of the three organizations; however, the goodness-of-fit test results were not satisfactory.
The inconsistent results of the LACE index in predicting the risk of death or 30-day readmission using our partners’ local data speak to the importance of careful model construction and rigorous testing prior to adopting a predictive model in a hospital setting. Our Research Collaborative has evolved to Phase II, Logibec’s BI team composed of statistical, clinical, and IT experts are currently developing and validating a customizable Readmission Risk Prediction Tool.
We have built the model using both traditional statistical methods and machine learning methods to predict readmission, and compared model discrimination and predictive range of the various techniques so as to build a model that delivers the highest level of accuracy while passing the various test criteria.
Being rigorous and cautious about the data chosen when building a prediction model is essential to its success when deployed in the real world!