Best Practice in the Assessment of Competence: A Literature Review
What were the key findings?
There were extensive findings, which included:
- There is no clear and unambiguous definition of ‘professionalism’. There is a set of constructs and behaviours which are associated with ‘professionalism’. It may be more useful to break down ‘professionalism’ into elements such as communication and situated judgements.
- Good practice is apparent in examples of all types of assessment, but the key decision for future implementation is around what is to be assessed – global judgements, specifically defined behaviours, or the ability to demonstrate a professional response.
- Examples of good practice in assessing ethics were demonstrated in concordance tests, where candidates’ decision making is compared to experts’.
- The use of simulation and virtual reality for assessing technical skills is well-established, but evidence of good practice remains mixed.
- Involving stakeholders, including patients, may help to define important performance criteria.
- There is evidence of differential attainment between home and international graduates when taking licensing examinations in different countries. Despite this, equality and diversity considerations are largely not addressed in research. Risks of bias in assessment design should be considered.
Further findings can be found in the full paper.
Why did we commission this research?
We are introducing the Medical Licensing Assessment (MLA), which will assess doctors’ readiness for licensed practice. As part of those preparations, we wanted to know what research existed about:
- Good practice in the use, or potential use of summative assessments around professionalism, ethics and competence in relation to patient safety.
- Evidence for the use of simulation or other technologically-mediated methods in summative assessments in medical and non-medical contexts.
The research specifically focussed on summative (end of course) assessments, as the MLA will be a summative assessment.
The research will feed into our thinking as we develop the MLA.
What did the research involve?
This was a literature review, where existing research was appraised to find information of relevance to the questions.
Medical and non-medical research databases were systematically searched to identify over 9900 papers describing assessments of professionalism, ethics and patient safety, or using novel approaches to assessment. These were then screened to confirm that they were relevant. The decision on whether they showed ‘good practice’ was made by considering the evidence provided of assessments’ validity (the degree to which the assessment measures what it is intended to measure). Based on this, a final set of 248 papers, 140 considered highly relevant and a further 108 less highly relevant, were studied to create the final report.