Evaluating the regulatory impact of medical revalidation

What were the key findings?

The report presents findings from the UK Medical Revalidation Evaluation Collaboration’s (UMbRELLA) three-year study into the impact of revalidation. The key findings section here is taken directly from the report and is organised in relation to the six research questions the study addressed. 

Is the GMC’s objective of bringing all doctors into a governed system that evaluates their fitness to practise on a regular basis being consistently achieved?

• Overall, most doctors have been brought into a governed system of medical revalidation.
• There are higher deferral rates in some groups, including female doctors, younger doctors and those from black and minority ethnic backgrounds independent of where they gained their primary medical qualification.
• Engagement in revalidation has generally been more straightforward for doctors working within existing governance structures, for example as an employee for one organisation.
• Medical revalidation has led to a rise in participation by UK doctors in annual appraisal.
• Outside existing governance structures, there are peripheral groups, including but not exclusively locums, where the ability to obtain an annual appraisal has been inconsistent.
• There are inconsistencies at the appraisal level for all doctors, where local and appraiser interpretations are central in shaping individual doctors’ experience of the system.

How is the requirement for all doctors to collect and reflect upon supporting information about their whole practice through appraisal being experienced by revalidation stakeholders? 

• Overall, doctors are able to collect the required supporting information.
• However, the ease with which doctors can collect some types of supporting information may vary according to their job role, setting or speciality.
• The requirement to submit supporting information across six defined categories during the five-year cycle has resulted in a strong focus within the appraisal process on the collection of SI.
• Doctors found patient and colleague feedback, and significant event analysis, most helpful in informing reflective discussions.
• Reflection on supporting information in appraisal is key for generating change, but reflection is often seen as just a product of appraisal, not necessarily translated into ongoing reflective practice.
• Expectations set locally, for example by employing organisations or individual appraisers, can influence doctors’ experiences of supporting information collection and can go beyond the requirements we set for revalidation.

Is engagement in revalidation promoting medical professionalism by increasing doctors’ awareness and adoption of the principles and values set out in Good medical practice? 

• A significant minority of doctors reported changing an aspect of their clinical practice, professional behaviour or learning activities as a result of their most recent appraisal.
• Overwhelmingly these changes related to the focus or quantity of their continuing professional development activities, though changes have occurred across the domains of Good medical practice.
• However, some doctors identified potentially negative impacts on practice or for professional autonomy.
• Revalidation, through appraisal, provides a means to document practice but may not necessarily improve professional practice.
• Ultimately, revalidation’s ability to promote good professional practice is through the central role of high quality formative appraisal.

Are revalidation mechanisms facilitating the identification and remedy of potential concerns before they become safety issues or fitness to practise referrals?

• Many in the profession believe that the main aim of revalidation is to identify ‘bad doctors’, and that doctors’ participation in appraisal will not achieve this aim.
• Since late 2012, fitness to practise referrals from employers have returned to pre-2009 levels, following spikes in such activity in the period leading up to revalidation’s introduction. There is no statistical evidence, as yet, that referrals from employers have dropped as a result of the earlier identification and local remedy of concerns.
• Appraisal and appraisers can and do identify some concerns about doctors, particularly in relation to workplace and health issues, and many concerns identified through appraisal are addressed successfully within that process.

How do responsible officers fulfil their statutory function of advising the GMC about doctors’ fitness to practise and what support do they have in this role? 

• Responsible officers’ approaches to decision-making vary in terms of the information that they use and the extent to which they delegate or share decision-making responsibility.
• The size of the organisation in which they work, particularly the number of doctors connected to it, is a key factor in shaping responsible officers’ approaches to decision-making.
• Some responsible officers do not feel that the three options available for revalidation recommendations (revalidate; deferral; and non-engagement) adequately cover all circumstances.
• Our Employer Liaison Service plays a key and developing role in supporting responsible officers and acting as the point of contact between organisations and the medical regulator.

Are patients being effectively and meaningfully engaged in revalidation processes? 

• Many of those involved in revalidation view patient and public involvement positively, but there is confusion over its intended purpose and appropriate modes of delivery.
• Both doctors’ and patients’ engagement with patient feedback is inconsistent and at times problematic. A need for current patient feedback tools to be refined was repeatedly expressed from both patient and doctor perspectives.
• Patient complaints and compliments can have a negative or positive impact on performance. More formal ways of providing compliments is desirable.
• Lay representation in revalidation processes has increased since its implementation but activity varies across organisations. Existing lay representatives have identified key ways in which lay roles could be developed and supported.

Why did we commission this research?

We want to understand the impact revalidation is having since we introduced it in 2012, learn from the experiences of those directly involved and understand whether we can make any improvements.

What did the research involve?

The UMbRELLA study consists of seven work packages, organised by research methods, and designed to collect and analyse quantitative and qualitative data across revalidation’s component activities. 

The completed study involved nine literature reviews, the analysis of pseudonymised data we held on nearly 281,000 doctors, and eight surveys with over 85,000 participants. 44 appraisals were recorded and analysed, 156 doctors and patient representatives interviewed and 24 doctors’ portfolios reviewed. 

Evaluating the regulatory impact of medical revalidation.