Direct Observation of Procedural Skills for Entry-Level Competency Assessment: A Pilot Study of Arterial Blood Gas sampling

##plugins.themes.academic_pro.article.sidebar##

Published Jul 12, 2026
https://doi.org/10.62438/tunismed.v104i07.6739
Download
PDF
Vol. 104 No. 07 (2026): Tunis Med Jul 2026

##plugins.themes.academic_pro.article.main##

Khedija Zaouche
Emergency Department, Mahmoud El Matri Hospital, Faculty of Medicine of Tunis, University of Tunis El Manar, Tunis, Tunisia
Aida Benzarti
Department of Occupational Medicine, la Rabta Hospital, Pedagogical Committee, Faculty of Medicine of Tunis, University of Tunis El Manar, Tunis, Tunisia
Fedia El Ayech
Emergency Department, Mahmoud El Matri Hospital, Faculty of Medicine of Tunis, University of Tunis El Manar, Tunis, Tunisia
Emna Rezgui
Emergency Department, Mahmoud El Matri Hospital, Faculty of Medicine of Tunis, University of Tunis El Manar, Tunis, Tunisia
Emna Kallel
Emergency Department, Mahmoud El Matri Hospital, Faculty of Medicine of Tunis, University of Tunis El Manar, Tunis, Tunisia
Manel Kallel
Emergency Department, Mahmoud El Matri Hospital, Faculty of Medicine of Tunis, University of Tunis El Manar, Tunis, Tunisia

Abstract

Introduction: Proficiency in arterial blood gas (ABG) sampling is expected of senior medical students, yet it is often assumed rather than objectively assessed. Early identification of procedural skill gaps is essential to ensure patient safety, particularly in emergency settings. This study aimed to evaluate the value of Direct Observation of Procedural Skills (DOPS) as a diagnostic tool at the start of an emergency medicine clerkship.
Methods: In this prospective pilot study, 21 fifth-year medical students were assessed while performing ABG sampling on real patients using a standardized DOPS checklist. The primary outcome was the level of supervision required. Secondary outcomes included performance in technical and non-technical domains, prior procedural experience, and self-reported stress.
Results: Sixteen students (76%) required direct supervision, while only one (5%) achieved near-independent performance. Technical skills were generally satisfactory; however, major deficiencies were observed in informed consent (71%) and communication (62%). Prior experience of ≥10 procedures was significantly associated with higher competence (p < 0.01), whereas no association was found with stress levels.
Conclusion: Procedural competence in ABG sampling is heterogeneous among senior students. Early implementation of DOPS helps identify gaps,
particularly in non-technical domains, and supports tailored supervision to enhance patient safety.

Keywords:

Clinical Competence, Undergraduate Medical Education, Feedback, Emergency Medicine

##plugins.themes.academic_pro.article.details##

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

References

  1. Monga S, Agarwal D. Evaluation of direct observation of procedural skills (DOPS) as an assessment method for MBBS students.
  2. Res J Med Sci. 2024;18:436–440. https://doi.org/10.36478/ makrjms.2024.12.436.440
  3. Cunanan B, Muppa H, Orellana MA, Bates RN, McGain F. Blood gas sampling in the intensive care unit: a prospective before-and-after
  4. interventional study on the effect of an educational program on blood gas testing frequency. Aust Crit Care. 2024;37(5):755–760.
  5. https://doi.org/10.1016/j.aucc.2024.01.009
  6. Erfani Khanghahi M, Ebadi Fard Azar F. Direct observation of procedural skills (DOPS) evaluation method: a systematic review
  7. of evidence. Med J Islam Repub Iran. 2018;32:45. https://doi. org/10.14196/mjiri.32.45
  8. Asadi K, Mirbolook AR, Haghighi M, Sedighinejad A, Naderi Nabi B, Abedi S, et al. Evaluation of satisfaction level of orthopedic interns
  9. from direct observation of procedural skills assessment (DOPS). Res Med Educ. 2012;4(2):17–23.
  10. Watson MJ, Wong DM, Kluger R, Chuan A, Herrick MD, Ng I, et al. Psychometric evaluation of a direct observation of procedural
  11. skills assessment tool for ultrasound-guided regional anaesthesia. Anaesthesia. 2014;69(6):604–612. https://doi.org/10.1111/
  12. anae.12625
  13. Cohen SN, Farrant PB, Taibjee SM. Assessing the assessments: UK dermatology trainees’ views of the workplace assessment tools. Br
  14. J Dermatol. 2009;161(1):34–39. https://doi.org/10.1111/j.1365- 2133.2009.09097.x
  15. Bindal N, Goodyear H, Bindal T, Wall D. Direct observation of procedural skills (DOPS) assessment: a study to evaluate the
  16. experience and opinions of trainees and assessors. Med Teach. 2013;35(6):e1230–e1234. https://doi.org/10.3109/014215
  17. X.2012.746447
  18. Dabaghi Tabriz F, Bahramian A, Asdagh S, Ghadimi F, Katebi K. The effect of direct observation of procedural skills on learning and
  19. satisfaction of students in a community dentistry course. J Adv Med Educ Prof. 2024;12(2):111–117. https://doi.org/10.30476/
  20. JAMP.2023.100431.1890
  21. Lagoo JY, Joshi SB. Introduction of direct observation of procedural skills (DOPS) as a formative assessment tool during postgraduate
  22. training in anaesthesiology: exploration of perceptions. Indian J Anaesth. 2021;65:202–209. https://doi.org/10.4103/ija.
  23. IJA_124_20
  24. Choudhary A, Gupta V. Teaching communications skills to medical students: introducing the fine art of medical practice. Int J Appl Basic Med Res. 2015;5(Suppl 1):S41–S44. https://doi.org/10.4103/2229-516X.162273
  25. Rela M, Price T. Review of the validity of direct observation of procedural skills (DOPS) as an assessment tool for the procedural
  26. skills of surgical trainees. Ann R Coll Surg Engl. 2023;105(7):599– 606. https://doi.org/10.1308/rcsann.2022.0052
  27. Yalcinturk AA, Ustun B. Medical errors room: the effect of simulation training on increasing awareness of nursing students about medical errors encountered in psychiatric clinics. Arch Psychiatr Nurs. 2023;43:37–42. https://doi.org/10.1016/j.apnu.2022.12.030
  28. Haldane T. "Portfolios" as a method of assessment in medical education. Gastroenterol Hepatol Bed Bench. 2014;7(2):89-93.
  29. Cobb K, Brown G, Jaarsma D, Hammond R. The educational impact of assessment: a comparison of direct observation of procedural
  30. skills and multiple-choice questions. Med Teach. 2013;35(11): e1598–e1607.
  31. https://doi.org/10.3109/0142159X.2013.803061
  32. Amini A, Shirzad F, Mohseni MA, Sadeghpour A, Elmi A. Designing direct observation of procedural skills (DOPS) test for selective
  33. skills of orthopedic residents and evaluating its effects from their points of view. Res Dev Med Educ. 2015;4(2):147-152. https://doi.
  34. org/10.15171/rdme.2015.026