Table 1 (abstract A4). Classification of best practices based on the stage of assessment implementation
From: Proceedings of the Saudi Health Simulation Conference 2018
| Phase | Examples of best practices |
|---|---|
| Planning phase | Right modality of simulation for the tasks to be evaluated |
| Real practice design to reflect the highest possible fidelity | |
| Appropriate structure and resources standards of the assessment centre | |
| Proper security measures to prevent breach of the exam content | |
| Clear and user friendly scenarios for SPs, examiners and technical support staff | |
| Determine the metrics of the assessment | |
| Carefully chosen/designed assessment tools (checklists, global ratings) to facilitate high standard metrics | |
| Raters should be trained and qualified based on a preset protocol | |
| Measures to assure high reliability and validity e.g. increasing the number of scenarios, designing task-specific stations and standardizing the exams administration | |
| Targeted metrics, e.g. Kappa >0.75, reliability, validity, accessibility, feasibility | |
| During the assessment phase | Timing each station 5-10 minutes, in acute care medicine can be shortened to 5 minutes |
| Video and audio recording, as a quality measure, to be considered | |
| Synchronizing and standardizing timing of the exam conducted in multiple centres | |
| Availability of real-time technical support in all centres | |
| Reinforcing measures to prevent cheating | |
| Room designation during the examination, i.e. examiner, SP and/or technician | |
| Post-assessment phase | Evaluation of the assessment process |
| Psychometric analysis of the scores to support the validity of the assessment | |
| Explore for confounding factors that influence the outcome | |
| Review the issues that appeared during the exam and actions accordingly | |
| Develop continuous improvement plan | |
| Update manuals and/or protocols | |
| Demonstrate results and appeal process to the learners | |
| Conduct remediation process as planned before |