University of Leicester
Browse

Individual participant data meta-analysis for external validation, recalibration, and updating of a flexible parametric prognostic model

Download (2.02 MB)
journal contribution
posted on 2021-10-01, 15:31 authored by J Ensor, KIE Snell, TPA Debray, PC Lambert, MP Look, MA Mamas, KGM Moons, RD Riley
Individual participant data (IPD) from multiple sources allows external validation of a prognostic model across multiple populations. Often this reveals poor calibration, potentially causing poor predictive performance in some populations. However, rather than discarding the model outright, it may be possible to modify the model to improve performance using recalibration techniques. We use IPD meta-analysis to identify the simplest method to achieve good model performance. We examine four options for recalibrating an existing time-to-event model across multiple populations: (i) shifting the baseline hazard by a constant, (ii) re-estimating the shape of the baseline hazard, (iii) adjusting the prognostic index as a whole, and (iv) adjusting individual predictor effects. For each strategy, IPD meta-analysis examines (heterogeneity in) model performance across populations. Additionally, the probability of achieving good performance in a new population can be calculated allowing ranking of recalibration methods. In an applied example, IPD meta-analysis reveals that the existing model had poor calibration in some populations, and large heterogeneity across populations. However, re-estimation of the intercept substantially improved the expected calibration in new populations, and reduced between-population heterogeneity. Comparing recalibration strategies showed that re-estimating both the magnitude and shape of the baseline hazard gave the highest predicted probability of good performance in a new population. In conclusion, IPD meta-analysis allows a prognostic model to be externally validated in multiple settings, and enables recalibration strategies to be compared and ranked to decide on the least aggressive recalibration strategy to achieve acceptable external model performance without discarding existing model information.

History

Citation

Statistics in Medicine. 2021;40,13:3066–3084.

Author affiliation

Department of Health Sciences

Version

  • VoR (Version of Record)

Published in

Statistics in Medicine

Volume

40

Issue

13

Pagination

3066-3084

Publisher

Wiley

issn

0277-6715

eissn

1097-0258

Acceptance date

2021-03-05

Copyright date

2021

Available date

2021-03-26

Language

English

Usage metrics

    University of Leicester Publications

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC