Evaluating the regulatory impact of medical revalidation
What were the key findings?
This report sets out findings from a three‑year study by the UK Medical Revalidation Evaluation Collaboration (UMbRELLA) on the impact of revalidation.
Is the GMC’s objective of bringing all doctors into a governed system that evaluates their fitness to practise on a regular basis being consistently achieved?
- Most doctors are now part of a formal system of medical revalidation.
- Some groups have higher rates of deferral. This includes female doctors, younger doctors, and doctors from black and minority ethnic backgrounds. These differences apply regardless of where doctors gained their primary medical qualification.
- Doctors working within established governance structures, such as being employed by a single organisation, have generally found it easier to take part in revalidation.
- Medical revalidation has increased participation by UK doctors in annual appraisal.
- Outside established governance structures, some groups have had less consistent access to annual appraisal. This includes, but is not limited to, locum doctors.
- There are also inconsistencies in how appraisals are carried out for all doctors. Local processes and individual appraiser interpretation play a key role in shaping doctors’ experiences of the system.
How is the requirement for all doctors to collect and reflect upon supporting information about their whole practice through appraisal being experienced by revalidation stakeholders?
- Overall, doctors are able to collect the supporting information they need.
- However, how easy this is can vary depending on a doctor’s role, work setting, or speciality.
- The requirement to submit supporting information across six defined categories over a five‑year cycle has led to a strong focus on collecting supporting information during appraisal.
- Doctors found patient feedback, colleague feedback, and significant event analysis the most useful for supporting reflective discussions.
- Reflection on supporting information during appraisal is important for driving change. However, reflection is often seen as something that happens only as part of appraisal, rather than as part of ongoing reflective practice.
- Local expectations, for example those set by employing organisations or individual appraisers, can shape doctors’ experiences of collecting supporting information. In some cases, these expectations go beyond the requirements set for revalidation.
Is engagement in revalidation promoting medical professionalism by increasing doctors’ awareness and adoption of the principles and values set out in Good medical practice?
- A significant minority of doctors reported making changes to their clinical practice, professional behaviour, or learning activities following their most recent appraisal.
- Most of these changes related to the focus or amount of continuing professional development activity. However, changes were reported across all domains of Good medical practice.
- Some doctors also reported potentially negative effects on their practice or on their professional autonomy.
- Revalidation, through appraisal, provides a way to record and evidence practice, but it does not always lead to improvements in professional practice.
- Ultimately, revalidation’s ability to support good professional practice depends on the quality of formative appraisal.
Are revalidation mechanisms facilitating the identification and remedy of potential concerns before they become safety issues or fitness to practise referrals?
- Many doctors believe that the main purpose of revalidation is to identify so‑called ‘bad doctors’. They also believe that taking part in appraisal is unlikely to achieve this.
- Since late 2012, referrals to fitness to practise from employers have returned to pre‑2009 levels. This followed a rise in referrals in the period leading up to the introduction of revalidation. There is currently no statistical evidence that employer referrals have fallen as a result of earlier identification and local resolution of concerns.
- Appraisal, and appraisers, can identify some concerns about doctors, particularly those related to workplace issues and health. Many of the concerns identified through appraisal are successfully addressed within that process.
How do responsible officers fulfil their statutory function of advising the GMC about doctors’ fitness to practise and what support do they have in this role?
- Responsible officers take different approaches to decision‑making. These differences relate to the information they use and how much decision‑making responsibility they delegate or share.
- The size of the organisation they work in, particularly the number of doctors connected to it, is a key factor in shaping how responsible officers make decisions.
- Some responsible officers feel that the three available revalidation recommendations (revalidate, deferral, and non‑engagement) do not fully cover all situations.
- Our Employer Liaison Service plays an important and developing role in supporting responsible officers. It also acts as the main point of contact between organisations and the medical regulator.
Are patients being effectively and meaningfully engaged in revalidation processes?
- Many people involved in revalidation view patient and public involvement positively. However, there is confusion about its intended purpose and the most appropriate ways to deliver it.
- Engagement with patient feedback is inconsistent for both doctors and patients, and can be problematic at times. Both groups have repeatedly highlighted the need to improve and refine current patient feedback tools.
- Patient complaints and compliments can have both negative and positive effects on performance. There is a desire for more formal ways to record and share compliments.
- Lay representation in revalidation processes has increased since revalidation was introduced, but levels of activity vary between organisations. Existing lay representatives have identified key ways in which lay roles could be further developed and better supported.
Why did we commission this research?
We want to understand the impact revalidation has had since it was introduced in 2012. Learn from the experiences of those directly involved, and identify where improvements can be made.
What did the research involve?
The UMbRELLA study is made up of seven work packages. These are organised by research method. They're designed to collect and analyse both quantitative and qualitative data across revalidation.
The completed study included nine literature reviews and analysis of pseudonymised data held on almost 281,000 doctors. It also involved eight surveys with more than 85,000 participants. In addition, 44 appraisals were recorded and analysed, 156 doctors and patient representatives were interviewed, and 24 doctors’ portfolios were reviewed.
