Do readers maintain word-level uncertainty during reading? A pre-registered replication study

We present a replication of Levy, Bicknell, Slattery, and Rayner (2009). In this prior study participants read sentences in which a perceptually confusable preposition (at; confusable with as) or non-confusable preposition (toward) was followed by a verb more likely to appear in the syntactic structure formed by replacing at with as (e.g. tossed) or a verb that was not more likely to appear in this structure (e.g. thrown). Readers experienced processing difficulty upon fixating verbs like tossed following at, but not toward. Levy et al. argued that this suggests readers maintained uncertainty about previously fixated words’ identities. We argue that this finding has wide-ranging implications for language processing theories, and that a replication is required. On the basis of a Bayes Factor Design Analysis we conducted a replication study with 56 items and 72 participants in order to determine whether Levy et al.’s effects are replicable. Using Bayesian statistical techniques we show that in our dataset there is evidence against the existence of the interaction Levy et al. found, and thus conclude that this study is non-replicable.