Assessing a quality assurance tool used to assess educator delivery of a structured self-management education programme: A Case Study

Background: Recommendations for assessing quality delivery of self-management support interventions for those with long-term conditions, like DESMOND in Type 2 Diabetes, encourage use of direct observation methods. Aim: Using the DESMOND programme as a case study, this study assessed aspects of effectiveness of the original assessment tool used to observe educator delivery. Method: A mixed methods approach was used to: (1) Assess the consistency of the original assessment tool with the programmes’ underlying theories and philosophy, (2) Develop a revised assessment tool suitable for assessing the delivery of DESMOND, (3) assess the reliability of the revised tool and (4) describe DESMOND delivery in relation to the revised tool. Results: (1) The original DESMOND assessment tool demonstrated good theoretical content validity. (2) The revised tool consisted of 39 core DESMOND and NONDESMOND congruent behaviours. (3) Inter-rater reliability of the tool was assessed as moderate. (4) Educators used a number of DESMOND congruent behaviours in their delivery of the programme, but also many NON-DESMOND congruent behaviours. The reasons for this were related to time, professional responsibilities and lack of confidence in knowing when and how to use the DESMOND behaviours. Conclusions: Current tools used to observe a nationwide structured self-management programme delivery are complex. A stepwise approach can be utilised to improve the validity of these and similar tools. Training needs of both assessors and educators can be identified using structured observation tools.