Accuracy of risk models used for public reporting of heart transplant center performance.

BACKGROUND: Heart transplant programs and regulatory entities require highly accurate performance metrics to support internal quality improvement activities and national oversight of transplant programs, respectively. We assessed the accuracy of publicly reported performance measures.

METHODS: We used the United Network for Organ Sharing registry to study patients who underwent heart transplantation between January 1, 2016 and June 30, 2018. We used tests of calibration to compare the observed rate of 1-year graft failure to the expected risk of 1-year graft failure, which was calculated for each recipient using the July 2019 method published by the Scientific Registry of Transplant Recipients (SRTR). The primary study outcome was the joint test of calibration, which accounts for both the total number of events predicted (calibration-in-the-large) and dispersion of risk predictions (calibration slope).

RESULTS: 6,528 heart transplants were analyzed. The primary test of calibration failed (p

CONCLUSIONS: Risk models employed for publicly-reported graft survival at U.S. heart transplant centers lack accuracy in general and in all subgroups tested. The use of disease-specific models may improve the accuracy of program performance metrics.