Measuring the anaesthesia clinical learning environment at the department level is feasible and reliable

Abstract

Background. The learning environment describes the context and culture in which trainees learn. In order to establish the feasibility and reliability of measuring the anaesthetic learning environment in individual departments we implemented a previously developed instrument in hospitals across New South Wales. Methods. We distributed the instrument to trainees from 25 anaesthesia departments and supplied summarized results to individual departments. Exploratory and confirmatory factor analyses were performed to assess internal structure validity and generalizability theory was used to calculate reliability. The number of trainees required for acceptable precision in results was determined using the standard error of measurement. Results. We received 172 responses (59% response rate). Suitable internal structure validity was confirmed. Measured reliability was acceptable (G-coefficient 0.69) with nine trainees per department. Eight trainees were required for a 95% confidence interval of plus or minus 0.25 in the mean total score. Eight trainees as assessors also allow a 95% confidence interval of approximately plus or minus 0.3 in the subscale mean scores. Results for individual departments varied, with scores below the expected level recorded on individual subscales, particularly the ‘teaching’ subscale. Conclusions. Our results confirm that, using this instrument, individual departments can obtain acceptable precision in results with achievable trainee numbers. Additionally, with the exception of departments with few trainees, implementation proved feasible across a training region. Repeated use would allow departments or accrediting bodies to monitor their individual learning environment and the impact of changes such as the introduction of new curricular elements, or local initiatives to improve trainee experience.