Table 4 Study ratings using Buckley's (modified) criteria
From: Observer roles that optimise learning in healthcare simulation education: a systematic review
| Criteria (Yes, No, Unclear) | Bell | Harder | Hober | Kaplan | Lau | Smith | Stegmann | Stiefel | Thidemann |
|---|---|---|---|---|---|---|---|---|---|
| Note: rater disagreement shown as two scores | |||||||||
| Clear research question | U | Y | Y | Y | U | Y | Y | U | Y |
| Subject group appropriate for study | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Reliable and valid methods (qualitative or quantitative) used | Y | Y | Y | Y | Y/U | Y | Y | Y | Y |
| Completeness of data (drop out, questionnaire response rate >60 %, attrition rate <50 %) | Y | Y | Y | Y | N | N | Y | N | Y |
| Controlled for confounders or acknowledged if non RCT design | N | U/N | N | U/N | U | N | Y | U | Y |
| Statistical and other analysis methods appropriate | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Data justifies the conclusions drawn | Y | Y | Y | Y | N | Y | Y | U/N | Y |
| Study could be replicated | Y/U | Y | Y | Y | U | N/U | Y | Y | Y |
| Prospective study | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Relevant ethical issues addressed | U | Y | Y | N | U | Y | Y | U | Y |
| Triangulation of data | Y | Y | Y | Y | N | Y | Y | N | Y |
| Total Score/11 (lowest score reported) | 7 | 10 | 10 | 9 | 3 | 8 | 11 | 5 | 11 |