Examining The Effect Of Test-Taker Personality On Performance In L2 Speaking Tests

<p dir="ltr">The increasing reliance on speaking exam results for critical decisions about test-takers highlights the importance of ensuring that these outcomes accurately and fairly reflect test-takers’ true abilities. This study adopts an interactionalist perspective, viewing paired speaking tests as co-constructed events between the two test-takers. Using an exploratory sequential mixed-methods design, the research seeks to deepen our understanding of the complexities surrounding the meaning of interactional competence (IC) scores derived from oral paired tests. Specifically, it investigates the influence of test-takers’ and their partners’ personality traits, namely ‘Agreeableness’, ‘Conscientiousness’, and ‘Openness to Experience’, on individual interactional discourse. This study also aims to strengthen the socio-cognitive framework for test validation (Chalahoub-Deville and O’Sullivan, 2020; Taylor, 2011; Weir, 2005), building on Messick’s (1989) theory of validity to advance our understanding of the complexities involved in deriving scores from L2 speaking tests. The quantitative phase of the study involved 120 Sri Lankan university undergraduates enrolled for the UTEL oral test. A Hierarchical Multiple Regression analysis revealed that ‘Gender’, ‘Test format’, and ‘Topic difficulty’ significantly explained 34.5% of the variance in IC scores (R² = 0.345), with significant changes in model fit in block 1 (F = 15.000, p < 0.001). However, ‘Agreeableness’, ‘Conscientiousness’, and ‘Openness to Experience’ showed no significant changes in blocks 2 and 3 (F = 0.272, p = 0.603 and F = 0.595, p = 0.619, respectively). Among the significant predictors, gender exhibited a negative relationship with IC scores, with females scoring 0.41 to 0.43 units lower on average. Test format had a positive effect on IC scores, increasing by 0.35 to 0.38 units, while topic difficulty emerged as the strongest predictor, with more challenging topics correlating with higher IC scores, ranging from 0.531 to 0.560. For the qualitative phase, trait agreeableness was chosen due to its stronger p-value. Test-takers’ (n = 24) UTEL live test performance was recorded, and their interactive task performance was transcribed for Conversation Analysis (CA). Of the 24 participants, 20 participated in the SRI session. CA analysis of the two personality pairings (heterogeneous and homogeneous) and language proficiency levels (B1 and C1) revealed distinct patterns. At the B1 level, homogeneous pairs exhibited simple, direct topic development and rigid turn-taking, with basic listener support, while heterogeneous pairs demonstrated more varied perspectives, less rigid turn-taking, and more engaged listener support. Similarly, at the C1 level, homogeneous pairs displayed structured but less complex topic development, smooth yet predictable turn-taking, and consistent but straightforward listener support, whereas heterogeneous pairs exhibited more flexible and expansive topic development, fluid and spontaneous turn-taking, and provided strategic and active listener support. The SRI findings further corroborated the CA results, emphasising the importance of collaboration, emotional regulation, adaptability, and pre-task planning. These factors are closely linked to the trait of agreeableness. These research findings enhance the understanding of the variables involved in the evaluation of paired speaking tasks, offering valuable insights for policymakers engaged in the UTEL test and other leading language testing organisations. Such insights may contribute to the development of more comprehensive and equitable assessment practices in second language (L2) paired speaking tests.</p>