University of Leicester
Determining the sample size required to establish whether a medical device is non-inferior to an external benchmark.pdf (638.32 kB)
Download file

Determining the sample size required to establish whether a medical device is non-inferior to an external benchmark

Download (638.32 kB)
journal contribution
posted on 2018-03-15, 14:40 authored by A Sayers, Michael J. Crowther, A Judge, M. R. Whitehouse, A. W. Blom
OBJECTIVES: The use of benchmarks to assess the performance of implants such as those used in arthroplasty surgery is a widespread practice. It provides surgeons, patients and regulatory authorities with the reassurance that implants used are safe and effective. However, it is not currently clear how or how many implants should be statistically compared with a benchmark to assess whether or not that implant is superior, equivalent, non-inferior or inferior to the performance benchmark of interest.We aim to describe the methods and sample size required to conduct a one-sample non-inferiority study of a medical device for the purposes of benchmarking. DESIGN: Simulation study. SETTING: Simulation study of a national register of medical devices. METHODS: We simulated data, with and without a non-informative competing risk, to represent an arthroplasty population and describe three methods of analysis (z-test, 1-Kaplan-Meier and competing risks) commonly used in surgical research. PRIMARY OUTCOME: We evaluate the performance of each method using power, bias, root-mean-square error, coverage and CI width. RESULTS: 1-Kaplan-Meier provides an unbiased estimate of implant net failure, which can be used to assess if a surgical device is non-inferior to an external benchmark. Small non-inferiority margins require significantly more individuals to be at risk compared with current benchmarking standards. CONCLUSION: A non-inferiority testing paradigm provides a useful framework for determining if an implant meets the required performance defined by an external benchmark. Current contemporary benchmarking standards have limited power to detect non-inferiority, and substantially larger samples sizes, in excess of 3200 procedures, are required to achieve a power greater than 60%. It is clear when benchmarking implant performance, net failure estimated using 1-KM is preferential to crude failure estimated by competing risk models.



BMJ Open, 2017, 7 (8), pp. e015397

Author affiliation

/Organisation/COLLEGE OF LIFE SCIENCES/School of Medicine/Department of Health Sciences


  • VoR (Version of Record)

Published in

BMJ Open


BMJ Publishing Group





Acceptance date


Copyright date


Available date


Publisher version



Usage metrics

    University of Leicester Publications