Measuring fidelity to extreme programming: a psychometric approach

This study assesses the Shodan survey as an instrument for measuring an individual’s or a team’s adherence to the extreme programming (XP) methodology. Specifically, we hypothesize that the adherence to the XP methodology is not a uni-dimensional construct as presented by the Shodan survey but a multidimensional one reflecting dimensions that are theoretically grounded in the XP literature. Using data from software engineers in the University of Sheffield’s Software Engineering Observatory, two different models were thus tested and compared using confirmatory factor analysis: a uni-dimensional model and a four-dimensional model. We also present an exploratory analysis of how these four dimensions affect students’ grades. The results indicate that the four-dimensional model fits the data better than the uni-dimensional one. Nevertheless, the analysis also uncovered flaws with the Shodan survey in terms of the reliability of the different dimensions. The exploratory analysis revealed that some of the XP dimensions had linear or curvilinear relationship with grades. Through validating the four-dimensional model of the Shodan survey this study highlights how psychometric techniques can be used to develop software engineering metrics of fidelity to agile or other software engineering methods.