NTS priority lists (private tool only)
NTS priorities list for training posts
The NTS priorities list is a different way of looking at the post specialty by site report that helps us prioritise which areas may need our support.
It uses four measures to identify the lowest scoring posts in the survey results
- most red flags
- most triple reds
- lowest average scores
- biggest drops in average score.
The score in each measure is calculated firstly as a proportion of possible outcomes, for example three red flags from a possible ten indicators would give 30%. This is to account for the fact that not every post will have a score in every indicator. That percentage is then compared to all other posts and ranked. The number displayed is the percentage of all posts that have a better score (known as percentile rank).
If the percentile rank is equal to or greater than the selected threshold (default 99.5%), then the post will appear on this list, highlighted in dark blue.
Finally, posts that are on the priority list, but that are highlighted in dark grey, are included in the list because there are fewer than five posts of that particular specialty in total. Because the low scores and changes in scores measures select the bottom 5% of posts in each indicator, posts in these smaller specialties appear more often on the list and should be interpreted with care.
Details about each of the four measures and other technical notes are provided below:
Measure details
Measure | Definition |
---|---|
Red flags | This measure tells us which posts have the most red flags in the current year. For each post, we calculate the percentage of possible indicator outcomes that are red and compare that percentage with all other posts. |
Changes in scores | This measure is designed to spot posts which have substantially lower mean scores compared to the previous year across many different indicators. We calculate the difference between the mean scores for last year and this year for every indicator for each post. We then compare each of these figures in groups of the same specialty and indicator. When a score is in the lowest 5% in any of those groups we flag it. We then use the same methodology to compare and rank posts as we do for the 'red flags' measure described above. Please note that two consecutive years of data are needed in a site for this measure to appear. Where a site’s code has changed from one year to the next, a comparison of its score between those years won’t be made and this measure won’t appear. This can happen to sites whose trusts have merged, for instance. |
Low scores | This measure is designed to spot posts that have low scores in different indicators, but aren't statistically significantly below the national average, meaning they don't have many red flags. We compare this years' mean scores in groups of the same specialty and indicator. When a score is in the lowest 5% in any of those groups we flag it. We then use the same methodology to compare and rank posts as we do for the 'red flags' measure described above. |
Triple reds | This measure is designed to find the posts that have the most ongoing poor survey results. For each post, we calculate the percentage of possible indicator outcomes that are red in each of the three previous years and compare that percentage with all other posts. Please note that three consecutive years of data are needed in a site for this measure to appear. Where a site’s code has changed from one year to the next, a comparison of red flags between those years won’t be made, meaning the triple red measure won’t appear. This can happen to sites whose trusts have merged, for instance. |
NTS priority list based on individual questions
This report highlights training sites and posts where respondents have reported particular concerns in response to six key questions in the NTS. These questions are designed to highlight specific issues around workload, reporting systems and clinical and education supervision that might not be apparent from the results of an indicator analysis alone.
Training environments that meet the criteria for one of the questions listed below may need our support to ensure that safe working practices are being followed and patient safety is assured.
Questions and criteria
Question master code | Question text | Priority list inclusion criteria |
---|---|---|
GENHQ155 and 156 | To what extent do you agree or disagree with the following statements? GENHQ155: My educational supervisor is easily accessible should I need to contact them. GENHQ156: The level of contact from my educational supervisor is appropriate for my training needs. |
Training organisations must ensure trainees have access to an educational supervisor (Promoting excellence R2.15). This report highlights all posts where at least two trainees have chosen 'Not applicable (I don't have an educational supervisor)' to either question. |
GENHQ184 | In this post how often (if at all) have you been asked to or felt pressured to submit a record of hours worked that were less than the hours you actually worked? | Multiple trainees choosing 'more than once' could indicate unsafe practices with regard to workload. This report highlights all posts where at least three trainees have chosen 'more than once'. |
GENHQ196 | In this post do you always know who is providing your clinical supervision when you're working? | Foundation doctors must at all times have on-site access to a senior colleague who is suitably qualified to deal with problems that may arise during the session. (Promoting excellence R1.8). This report highlights all posts where a foundation trainee has chosen 'No, there is no one I can contact'. |
GENHQ206 | In this post, out of hours, do you always know who is providing your clinical supervision when you're working? | Foundation doctors must at all times have on-site access to a senior colleague who is suitably qualified to deal with problems that may arise during the session. (Promoting excellence R1.8). This report highlights all posts where one or more foundation trainees have chosen 'No, there is no one I can contact'. |
GENHQ100 | To what extent do you agree or disagree with the following statement? I have been made aware of how to report patient safety incidents and near misses. |
Multiple respondents choosing 'Strongly disagree' to this question may indicate an unsafe culture in the training environment. This report highlights sites where multiple doctors in training (representing at least 10% of all doctors in training at the site) disagreed or strongly disagreed that they were made aware of how to report safety incidents and near misses. |