Impact of MCQ-Writing Training on the Psychometric Quality of a Surgery Examination: A Before-and-After Study

##plugins.themes.academic_pro.article.sidebar##

Published Oct 11, 2026
https://doi.org/10.62438/tunismed.v104i09.7053
Download
PDF
Vol. 104 No. 09 (2026): Tunis Med Sept 2026

##plugins.themes.academic_pro.article.main##

Baha Khlifa
Urology Department, Ibn El Jazzar University Hospital, Kairouan, Ibn El Jazzar Street, 3100 Kairouan. Ibn El Jazzar Faculty of Medicine, Sousse, Mohamed Karoui Street, 4002 Sousse, Tunisia
Rayen Lahouar
Urology Department, Ibn El Jazzar University Hospital, Kairouan, Ibn El Jazzar Street, 3100 Kairouan. Ibn El Jazzar Faculty of Medicine, Sousse, Mohamed Karoui Street, 4002 Sousse, Tunisia
Jihène Krichène
Urology Department, Ibn El Jazzar University Hospital, Kairouan, Ibn El Jazzar Street, 3100 Kairouan. Ibn El Jazzar Faculty of Medicine, Sousse, Mohamed Karoui Street, 4002 Sousse, Tunisia
Mohamed Kahloul
Department of Anesthesiology and Intensive Care, Sahloul University Hospital, 4011, Sousse. Ibn El Jazzar Faculty of Medicine, Sousse, Rue Mohamed Karoui, 4002 Sousse, Tunisia
Badreddine Ben Khalifa
Urology Department, Ibn El Jazzar University Hospital, Kairouan, Ibn El Jazzar Street, 3100 Kairouan. Ibn El Jazzar Faculty of Medicine, Sousse, Mohamed Karoui Street, 4002 Sousse, Tunisia
Wael Gazzah
Urology Department, Ibn El Jazzar University Hospital, Kairouan, Ibn El Jazzar Street, 3100 Kairouan. Ibn El Jazzar Faculty of Medicine, Sousse, Mohamed Karoui Street, 4002 Sousse, Tunisia
Sahbi Naouar
Urology Department, Ibn El Jazzar University Hospital, Kairouan, Ibn El Jazzar Street, 3100 Kairouan. Ibn El Jazzar Faculty of Medicine, Sousse, Mohamed Karoui Street, 4002 Sousse, Tunisia
Braiek Salem
Urology Department, Ibn El Jazzar University Hospital, Kairouan, Ibn El Jazzar Street, 3100 Kairouan. Ibn El Jazzar Faculty of Medicine, Sousse, Mohamed Karoui Street, 4002 Sousse, Tunisia

Abstract

Introduction: Summative assessment plays a central role in medical education and requires valid and reliable tools. The psychometric quality of multiple-choice questions (MCQs) largely depends on their design.
Objective: To assess the impact of pedagogical training in MCQ writing on the psychometric properties of a surgery examination at the Faculty of Medicine of Sousse.
Methods: This was a pre-experimental before-and-after study comparing two successive sessions of the same examination, conducted in 2024 and 2025. A total of 174 students, including 96 in period 1 and 78 in period 2, as well as 180 items, were analyzed. The assessed parameters
included the difficulty index, discrimination index, effective coefficient, response categories, and global test indicators, particularly Cronbach’s alpha and measurement error.
Results: Overall student performance, in terms of mean score and success rate, was comparable between the two periods (p = 0.774). However, the median discrimination index significantly improved, increasing from 28.27 to 40% (p = 0.003). The proportion of items with excellent
discrimination increased from 23.3 to 50% (p < 0.001). Cronbach’s alpha increased from 0.87 to 0.93, while the difficulty index remained stable.
Conclusion: Pedagogical training improved the psychometric quality of the examination without changing overall student performance, highlighting the value of an institutional approach based on continuous faculty training and regular psychometric analysis of examinations.

Keywords:

Certifying assessment, multiple-choice questions, item analysis, discrimination index, faculty development, medical education

##plugins.themes.academic_pro.article.details##

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

References

  1. Kassam A, de Vries I, Zabar S, Durning SJ, Holmboe E, Hodges B, et al. The Next Era of Assessment Within Medical Education: Exploring Intersections of Context and Implementation. Perspect Med Educ. 2024;13(1):496-506. doi:10.5334/pme.1128 PubMed PMID:
  2. ; PubMed Central PMCID: PMC11469546.
  3. Zubairi NA, AlAhmadi TS, Ibrahim MH, Hegazi MA, Gadi FU. Effective use of Item Analysis to improve the Reliability and Validity
  4. of Undergraduate Medical Examinations: Evaluating the same exam over many years: a different approach. Pak J Med Sci. mars
  5. ;41(3):810-5. doi:10.12669/pjms.41.3.10693 PubMed PMID: 40103890; PubMed Central PMCID: PMC11911747.
  6. Eleragi AMS, Miskeen E, Hussein K, Rezigalla AA, Adam MIE, Al-Faifi JA, et al. Evaluating the multiple-choice questions quality at the
  7. College of Medicine, University of Bisha, Saudi Arabia: a three-year experience. BMC Med Educ. 13 févr 2025;25(1):233. doi:10.1186/
  8. s12909-025-06700-2 PubMed PMID: 39948528; PubMed Central PMCID: PMC11827222.
  9. Rashwan NI, Aref SR, Nayel OA, Rizk MH. Postexamination item analysis of undergraduate pediatric multiple-choice questions
  10. exam: implications for developing a validated question Bank. BMC Med Educ. 21 févr 2024;24(1):168. doi:10.1186/s12909-
  11. -05153-3 PubMed PMID: 38383427; PubMed Central PMCID: PMC10882907.
  12. Hye T, Hossain MF, Roni MA. Interventions to Improve the Quality of Multiple-Choice Questions: A Systematic Review and Meta-
  13. Analysis. Am J Pharm Educ. nov 2025;89(11):101872. doi:10.1016/j. ajpe.2025.101872 PubMed PMID: 40962033.
  14. Betts J, Muntean W, Kim D, Kao S chuan. Evaluating Different Scoring Methods for Multiple Response Items Providing
  15. Partial Credit. Educ Psychol Meas. févr 2022;82(1):151-76. doi:10.1177/0013164421994636 PubMed PMID: 34992310;
  16. PubMed Central PMCID: PMC8725057.
  17. Masters GN. A rasch model for partial credit scoring. Psychometrika. 1 juin 1982;47(2):149-74. doi:10.1007/BF02296272
  18. Kim HY. Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor Dent Endod.
  19. févr 2013;38(1):52-4. doi:10.5395/rde.2013.38.1.52 PubMed PMID: 23495371; PubMed Central PMCID: PMC3591587.
  20. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 27 juin 2011;2:53-5. doi:10.5116/ijme.4dfb.8dfd PubMed
  21. PMID: 28029643; PubMed Central PMCID: PMC4205511.
  22. Tavakol M, Brennan RL. Medical education assessment: a brief overview of concepts in generalizability theory. Int J Med Educ. 11
  23. sept 2013;4:221-2. doi:10.5116/ijme.5278.a850 PubMed PMID: 28035055; PubMed Central PMCID: PMC4205529.
  24. Sadeghi P, Pourabbas A, Dehghani G, Katebi K. Quantitative and qualitative item analysis of exams of basic medical sciences
  25. departments of Tabriz University of Medical Sciences in 2023. BMC Med Educ. 1 juill 2025;25(1):937. doi:10.1186/s12909-025-07539-3
  26. PubMed PMID: 40597222; PubMed Central PMCID: PMC12211469.
  27. Hermi A, Achour W. Item analysis of examinations in the Faculty of Medicine of Tunis. Tunis Med. avr 2016;94(4):247-52. PubMed
  28. PMID: 27704506.
  29. Ayub R, Kiran F, Shabnam N, Rauf A, Fatima F. Effects of a long term faculty development program on improvement in quality of MCQs: an impact evaluation study. BMC Med Educ. 15 avr 2025;25:541. doi:10.1186/s12909-025-07081-2 PubMed PMID: 40234907;
  30. PubMed Central PMCID: PMC12001514.
  31. Owolabi LF, Adamu B, Taura MG, Isa AI, Jibo AM, Abdul-Razek R, et al. Impact of a longitudinal faculty development program on
  32. the quality of multiple-choice question item writing in medical education. Ann Afr Med. 2021;20(1):46-51. doi:10.4103/aam.
  33. aam_14_20 PubMed PMID: 33727512; PubMed Central PMCID: PMC8102895.
  34. Abdulghani HM, Ahmad F, Irshad M, Khalil MS, Al-Shaikh GK, Syed S, et al. Faculty development programs improve the quality
  35. of Multiple Choice Questions items’ writing. Sci Rep. 1 avr 2015;5:9556. doi:10.1038/srep09556 PubMed PMID: 25828516;
  36. PubMed Central PMCID: PMC4381327.
  37. Gupta P, Meena P, Khan AM, Malhotra RK, Singh T. Effect of Faculty Training on Quality of Multiple-Choice Questions. Int J Appl Basic
  38. Med Res. 2020;10(3):210-4. doi:10.4103/ijabmr.IJABMR_30_20 PubMed PMID: 33088746; PubMed Central PMCID: PMC7534721.
  39. Dellinges MA, Curtis DA. Will a Short Training Session Improve Multiple-Choice Item-Writing Quality by Dental School Faculty? A
  40. Pilot Study. J Dent Educ. août 2017;81(8):948-55. doi:10.21815/ JDE.017.047 PubMed PMID: 28765439.
  41. Haladyna TM, Downing SM, Rodriguez MC. A Review of Multiple- Choice Item-Writing Guidelines for Classroom Assessment.
  42. Appl Meas Educ. juill 2002;15(3):309-33. doi:10.1207/S15324818AME1503_5
  43. Pais J, Silva A, Guimarães B, Povo A, Coelho E, Silva-Pereira F, et al. Do item-writing flaws reduce examinations psychometric
  44. quality? BMC Res Notes. 11 août 2016;9:399. doi:10.1186/s13104-016-2202-4 PubMed PMID: 27516160; PubMed Central PMCID:
  45. PMC4982015.
  46. Shahat KA. Item Analysis of Multiple-Choice Question (MCQ)-Based Exam Efficiency Among Postgraduate Pediatric Medical Students:
  47. An Observational, Cross-Sectional Study From Saudi Arabia. Cureus. sept 2024;16(9):e69151. doi:10.7759/cureus.69151 PubMed
  48. PMID: 39262935; PubMed Central PMCID: PMC11388033.
  49. Bhattacherjee S, Mukherjee A, Bhandari K, Rout AJ. Evaluation of Multiple-Choice Questions by Item Analysis, from an Online
  50. Internal Assessment of 6th Semester Medical Students in a Rural Medical College, West Bengal. Indian J Community Med Off Publ
  51. Indian Assoc Prev Soc Med. 2022;47(1):92-5. doi:10.4103/ijcm. ijcm_1156_21 PubMed PMID: 35368481; PubMed Central PMCID:
  52. PMC8971860.
  53. Rezigalla AA, Eleragi AMESA, Elhussein AB, Alfaifi J, ALGhamdi MA, Al Ameer AY, et al. Item analysis: the impact of distractor efficiency
  54. on the difficulty index and discrimination power of multiple-choice items. BMC Med Educ. 24 avr 2024;24(1):445. doi:10.1186/s12909-
  55. -05433-y PubMed PMID: 38658912; PubMed Central PMCID: PMC11040895.
  56. Abdulghani HM, Irshad M, Haque S, Ahmad T, Sattar K, Khalil MS. Effectiveness of longitudinal faculty development programs on MCQs items writing skills: A follow-up study. PLOS ONE. 10 oct 2017;12(10):e0185895. doi:10.1371/journal.pone.0185895
  57. Abozaid H, Park YS, Tekian A. Peer review improves psychometric characteristics of multiple choice questions. Med Teach. avr
  58. ;39(sup1):S50-4. doi:10.1080/0142159X.2016.1254743 PubMed PMID: 28103718.
  59. Karthikeyan S, O’Connor E, Hu W. Barriers and facilitators to writing quality items for medical school assessments - a scoping review.
  60. BMC Med Educ. 2 mai 2019;19(1):123. doi:10.1186/s12909-019-1544-8 PubMed PMID: 31046744; PubMed Central PMCID:
  61. PMC6498649.
  62. Ali K, Zahra D. Ten tips for effective use and quality assurance of multiple-choice questions in knowledge-based assessments. Eur
  63. J Dent Educ Off J Assoc Dent Educ Eur. mai 2024;28(2):655-62. doi:10.1111/eje.12992 PubMed PMID: 38282273.
  64. Keskes S, Ghazali H, Chermiti I, Ben Turkia H, Ben Salah N, Boujelbene N, et al. Analyse docimologique des grilles critériées
  65. dans une épreuve d’examen clinique objectif structuré en médecine d’urgence : évaluation de la validité et de la fiabilité. Tunis Med.
  66. ;104(2):281-287. doi:10.62438/tunismed.v104i02.6440.