University of Leicester
Browse

Quantitative assessment of inconsistency in meta-analysis using decision thresholds with two new indices

journal contribution
posted on 2025-04-15, 15:55 authored by Bernardo Sousa-Pinto, Ignacio Neumann, Rafael José Vieira, Antonio Bognanni, Manuel Marques-Cruz, Sara Gil-Mata, Simone Mordue, Clareece NevillClareece Nevill, Gianluca Baio, Paul Whaley, Guido Schwarzer, James Steele, Gavin Stewart, Holger J Schünemann, Luís Filipe Azevedo
Objectives: In evidence synthesis, inconsistency is typically assessed visually and with the I2 and the Q statistics. However, these measures have important limitations (i) if there are few primary studies of small sample sizes or (ii) if there are multiple studies with precise estimates. In addition, with the increasing use of decision thresholds (DT), for example in Grading of Recommendations Assessment, Development and Evaluation evidence to decision (EtD) frameworks, inconsistency judgments can be anchored around DTs. In this article, we developed quantitative measures to assess inconsistency based on DTs. Study Design and Setting: We developed two measures to quantify inconsistency based on DTs – the decision inconsistency (DI) and the across-studies inconsistency (ASI) indices. The DI and the ASI are based on the distribution of the posterior samples studies’ effect sizes (ES) across interpretation categories defined by DTs. We developed these indices for the Bayesian context, followed by a frequentist extension. Results: The DI informs on the overall inconsistency of ESs across interpretation categories, while the ASI quantifies how different studies are compared to each other (in relation to interpretation categories) based on absolute effects. A DI ≥ 50% and an ASI ≥ 25% are suggestive of important inconsistency. We provide an R package (metainc) and a web tool (https://metainc.med.up.pt/) to support the computation of the DI and ASI, including in the context of sensitivity analyses assessing the impact of potential uncertainty in inconsistency. Conclusion: The DI and the ASI can contribute to quantitatively assess inconsistency, particularly as DTs are gaining recognition in evidence synthesis and health decision-making.

History

Author affiliation

College of Life Sciences Population Health Sciences

Version

  • VoR (Version of Record)

Published in

Journal of Clinical Epidemiology

Volume

181

Pagination

111725 - 111725

Publisher

Elsevier BV

issn

0895-4356

eissn

1878-5921

Copyright date

2025

Spatial coverage

United States

Language

en

Deposited by

Mrs Clareece Nevill

Deposit date

2025-04-08

Usage metrics

    University of Leicester Publications

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC