University of Leicester
Browse

Reporting and interpreting non-significant results in animal cognition research

Download (1.34 MB)
journal contribution
posted on 2023-04-14, 13:44 authored by Benjamin G Farrar, Alizée Vernouillet, Elias Garcia-Pelegrin, Edward W Legg, Katharina F Brecht, Poppy J Lambert, Mahmoud Elsherif, Shannon Francis, Laurie O’Neill, Nicola S Clayton, Ljerka Ostojić
How statistically non-significant results are reported and interpreted following null hypothesis significance testing is often criticized. This issue is important for animal cognition research because studies in the field are often underpowered to detect theoretically meaningful effect sizes, i.e., often produce non-significant p-values even when the null hypothesis is incorrect. Thus, we manually extracted and classified how researchers report and interpret non-significant p-values and examined the p-value distribution of these non-significant results across published articles in animal cognition and related fields. We found a large amount of heterogeneity in how researchers report statistically non-significant p-values in the result sections of articles, and how they interpret them in the titles and abstracts. Reporting of the non-significant results as “No Effect” was common in the titles (84%), abstracts (64%), and results sections (41%) of papers, whereas reporting of the results as “Non-Significant” was less common in the titles (0%) and abstracts (26%), but was present in the results (52%). Discussions of effect sizes were rare (<5% of articles). A p-value distribution analysis was consistent with research being performed with low power of statistical tests to detect effect sizes of interest. These findings suggest that researchers in animal cognition should pay close attention to the evidence used to support claims of absence of effects in the literature, and—in their own work—report statistically non-significant results clearly and formally correct, as well as use more formal methods of assessing evidence against theoretical predictions.

Funding

Benjamin G. Farrar was supported by the University of Cambridge BBSRC Doctoral Training Programme (BB/M011194/1). Alizée Vernouillet is currently supported by a BOF fellowship (BOF.PDO.2021.0035.01). Katharina F. Brecht was supported by a DFG Grant (BR 5908/1-1) and by a University of Tübingen Athene Fellowship. Mahmoud Elsherif is currently supported by The Baily Thomas Charitable Fund (TRUST/VC/AC/SG/5843-8995). Edward W. Legg is supported by a MSCA Fellowship (INTOM-794270).

History

Citation

Farrar BG, Vernouillet A, Garcia-Pelegrin E, Legg EW, Brecht KF, Lambert PJ, Elsherif M, Francis S, O’Neill L, Clayton NS, Ostojić L. 2023. Reporting and interpreting non-significant results in animal cognition research. PeerJ 11:e14963 https://doi.org/10.7717/peerj.14963

Author affiliation

School of Psychology and Vision Sciences

Version

  • VoR (Version of Record)

Published in

PeerJ

Volume

11

Pagination

e14963

Publisher

PeerJ

eissn

2167-8359

Acceptance date

2023-02-06

Copyright date

2023

Available date

2023-04-14

Language

en

Usage metrics

    University of Leicester Publications

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC