Survey development project
The 2012 survey will benefit from hard work on four elements of the NTS. Since early 2011, we have been working to improve the way we collect and validate survey data, the questionnaire and survey delivery mechanism, the survey management tool, and the survey reporting tool.
Menu
System improvements
We have moved the whole survey system into our core IT system, whereas previously a number of third party suppliers provided the data collection, survey management, survey delivery and reporting tools.
Trainees will complete their survey in GMC Online
Doctors use GMC Online to apply for registration, manage their contact details and pay their fees, among other things. Moving the survey into GMC Online will mean that trainees do not need to log into a separate system and do not need to wait for an invitation to begin the survey.
Once they have completed the survey, trainees will be able to log in at any time to see their completion code.
Please read our data protection statement for details about how we will use trainees’ training data and survey response data. (Coming soon)
You can also read more about this in our information for trainees section.
Deaneries will manage their survey records in GMC Connect
Deaneries use GMC Connect, our secure information portal, to manage programme approvals, GP trainer approvals and out of programme training approvals. They also use it to submit annual deanery reports and other returns.
By moving the national training survey records into GMC Connect we are meeting our aim of a single system approach for all data interaction between us and deaneries.
Read more about survey management in our information for deaneries section.
Survey reporting will be delivered through an Oracle business intelligence reporting solution
This means that reporting will be quicker, more flexible, more user driven and easier to navigate. Because we have integrated the reporting tool into our core system, reports will be available as soon as the survey closes (subject to a few weeks of checking the data is correct).
Users will be able to use the filters to create bespoke reports (such as comparing the results of two hospitals, how educational supervision differs between the surgical specialties, or look at handover across the four countries).
Read more about reporting in 2012 in our survey results section. (Coming soon)
Other improvements
Data
Meanwhile we have been working with deaneries to improve the reference data for the survey, making sure that all programmes and locations used for reporting match our approvals data. We have significantly overhauled data validation. A downloadable tool gives each deanery the ability to carry out sophisticated validation checks on their survey data before they send it to us, ensuring all of the records that get loaded into the survey are valid.
You can read about this year's data values in briefing note 4.
One change for 2012 will be that we will run the survey through GMC Online. For 2012, as with previous years, we will aim to collect survey participant data directly from each deanery. This data will then be validated before the survey opens. Before they can proceed with the survey, participants will be asked to verify the information which has been provided by their deaneries.
Questions
Generic questions
We have also improved the questionnaire for 2012. Much of the feedback we received suggested that the survey was too long and too difficult to complete. Among other improvements, we’ve halved the size of the survey for most participants.
Another key piece of feedback was that any review we carried out should be conducted with academic rigour, by survey experts. We strongly agreed with this principle and engaged experts from NHS Education for Scotland (NES) to carry out the review.
You can read about the changes we’ve made to the questionnaire in briefing note 5 and see the full question set in annex A to briefing note 5.
You can read the experts’ findings in the NHS Education for Scotland National Training Survey report 2012 (please note this file is 7mb).
Specialty specific questions
Like the generic questions, we have halved the total number of specialty specific questions in the survey this year. Most trainees will answer around 10 specialty specific questions.
The specialty specific questions complement the generic questions and are pertinent to the delivery of the respective curriculum. The survey is an ideal opportunity for Royal Colleges and Faculties to capture specific data on trainees’ perceptions of training.
The Royal Colleges and Faculties have supplied specialty specific questions for each specialty through a formal review process. Each question has been checked against good practice guidelines provided by our survey experts at NES.
You can read about the process for updating the specialty specific questions in briefing note 1.
You can read the good practice in questionnaire design guidance in briefing note 3.
Engagement
Between May and August 2011, we engaged with most of the key stakeholders in the survey including all the postgraduate deaneries, most of the Royal Colleges and Faculties (or Joint Committees where appropriate) and the BMA Junior Doctors’ Committee. We held an initial meeting with each to introduce them to the project, gauge perceptions and opinions of the survey.
You can read a summary of our findings from these meetings in Survey development briefing note 2.
During February 2012 we presented some of our work in our reporting roadshows in London, Manchester and Edinburgh. We showcased the new reporting tool, presented our work on the questionnaire for 2012 and showed our new approach to data collection.