Validity and reliability of an objective structured assessment tool for performance of ultrasound-guided regional anaesthesia

Shafqat, Atif, Rafi, M., Thanawala, Vishal, Bedforth, Nigel M., Hardman, J.G. and McCahon, Robert A. (2018) Validity and reliability of an objective structured assessment tool for performance of ultrasound-guided regional anaesthesia. British Journal of Anaesthesia, 121 (4). pp. 867-875. ISSN 1471-6771

PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Available under Licence Creative Commons Attribution Non-commercial No Derivatives.
Download (386kB) | Preview


Background: We aimed to examine the validity and reliability of previously developed criterion-referenced assessment checklist (AC) and global rating scale (GRS) to assess performance in ultrasound-guided regional anaesthesia (UGRA).

Methods: Twenty-one anaesthetists’ single, real-time, UGRA procedures (total: 21 blocks) were assessed using 22-item AC and 9-item GRS scored on a 3-point and 5-point Likert scales respectively. We used one-way ANOVA to compare assessment scores between 3 groups (group I: ≤30 blocks in the preceding year; group II: 31–100; group III: >100). Concurrent validity was evaluated using Pearson’s correlation (r). We calculated type-A intra- class correlation coefficient (ICC) using an absolute agreement definition in two-way random effects model, and inter-rater reliability (IRR) using absolute agreement between raters. The inter-item consistency was assessed by Cronbach’s alpha (α).

Results: Greater UGRA experience in the preceding year was associated with better AC [F (2,18) 12.01; p <0.001] and GRS [F (2,18) 7.44; p =0.004] scores. There was strong correlation between mean AC and GRS scores [r=0.73 (p <0.001)] and strong inter-item consistency for AC (α = 0.94) and GRS (α = 0.83). The ICC (95% CI) and IRR (95% CI) for AC was 0.96 (0.95 – 0.96) and 0.91 (0.88 – 0.95) respectively and 0.93 (0.90 – 0.94) and 0.80 (0.74 – 0.86) for GRS.

Conclusions: Both assessments differentiated between individuals who have performed fewer (≤30) and many (>100) blocks in the preceding year, supporting construct validity. It also established concurrent validity and overall reliability. We recommend both tools may be used in UGRA assessment.

Item Type: Article
Keywords: Anaesthetists; Checklist; Educational assessment; Reproducibility of results; Ultrasound
Schools/Departments: University of Nottingham, UK > Faculty of Medicine and Health Sciences > School of Medicine > Division of Clinical Neuroscience
Depositing User: Eprints, Support
Date Deposited: 27 Jun 2018 12:17
Last Modified: 31 Jul 2019 04:30

Actions (Archive Staff Only)

Edit View Edit View