Quality Panels 

Background

With the establishment of Health Education England (HEE) it became necessary to harmonise the quality processes of postgraduate medical education within this geographical area. These were previously the responsibility of two separate organisations, Severn Deanery and Peninsula Deanery. The quality teams from these organisations have produced a Quality Framework to encompass the whole of Postgraduate Medical Education within HEE, working across the South West. Quality Panels are an integral part of this framework. They have been in existence within the Severn geographical area for a few years. They have provided important information in respect of quality management of the programmes hosted by Severn. The harmonisation process has allowed an opportunity to review their current function and processes. This review is aimed at ensuring consistency of scrutiny and outcomes across programmes and specialties hosted within HEE, working across the South West.

This document describes the outcome of the review undertaken by the quality team (SPME and PPME). It is important that the functions and purpose of the Quality Panel is seen in the context of the overarching Quality Framework within which it is embedded.

Aim of Quality Panel

The aim of a Quality Panel is to provide improvement in the quality of postgraduate medical education. This will be achieved by an annual quality review of all the posts contained with the approved GMC training programme under assessment. 

What is a Quality Panel?

A quality panel is an annual meeting of selected individuals involved in the training programme under discussion. It has the remit to evaluate all the composite posts within the programme. It has a necessity to discuss information as described in this document. It has a mandate to produce an outcome for each post. It must restrict its comments to the posts and not the post holders. The composition, information to discuss and output is described later in this document.

There is acknowledgement that the programmes for which quality panels should run vary in their post composition, locality over which they run and specialty requirements that they should consider. The composition of the panel and its discussions should reflect this. The data considered and outcome of panels must have consistency across the whole of HEE, working across the South West, for our quality processes to be robust and credible. 

The Quality Panel Process

Each training programme will run an annual quality programme.



Timing

The quality panel will run during the period of July to December of the calendar year being considered.

Quality Panel Members

The quality panel will be chaired by the Training Programme Director (TPD) or equivalent e.g Specialty Tutor. There will be Lay Representation on most Quality Panels; this will depend upon availability and resource and Lay Representation will be audited by the Quality Team.

The specialty will decide the number and roles of other panel members. There must be trainee representation and the number of trainees on the panel must exceed the number of other panel members including the Chair.

The trainee representatives should have been allocated specific posts within the programme for them to report upon. This must be clear to the trainee at the time they are recruited to the panel. It is anticipated that the other non-trainee representatives will be a selection of accredited Educational Supervisors from the programme being evaluated.

External representation will occur at either the request of the panel Chair or the Quality Team.
A Head of School, Associate Dean or Quality Team member may attend the panel at the request of the panel members or in quality assurance capacity.

Information for Quality Panel to consider

The Quality Panel Chair and members can consider any information that they deem pertinent to inform their discussion. They must as a minimum consider the following information.

They must not discuss the individual performance of named trainees.

  • That calendar year’s GMC National Trainee Survey data
  • Reports from previous year’s Quality Panel(s) and/or LEP school visit(s)
  • Reports from Response to Concern (Triggered) visit(s) if applicable
  • Summary ARCP data for previous 12 months
  • Supervisor (Trainer) accreditation data
  • Quality Panel Score (Appendix 2)
  • Recruitment figures for Programme 

Quality Panel scoring process

In order to improve the objectivity and consistency of Quality Panel scoring the Quality team have developed a scoring tool which will enable training doctors to complete a scoring matrix prior to the quality panel. 

The Quality of each post will be considered from the perspective of providing a ‘Safe working Environment’ and  ‘Effective Educational Environment’. In each of these 2 domains the posts will be graded as either Excellent, Good, Requires Improvement or Inadequate.

The scoring matrices are for trainees to score the ‘Safe working environment’ and ‘Effective Educational Environment’ according to these grades. The matrices include brief descriptions defining Excellent, Good, Requires Improvement and Inadequate in a number of items that contribute to the quality of training and education. These items are reflective of the GMC requirements necessary to compliant with the GMC prescribed standards. Trainees will be asked to complete the scoring matrices using the principle of best fit to reach an overall grade for a post’s ‘Safe working Environment’ and ‘Effective Educational Environment’.

For example of the 10 items contributing to the ‘Safe working Environment’ score, if 2 were rated excellent, 6 good and 2 inadequate the overall ‘Safe working Environment’ score would be rated as good. Exceptions can be made to this, for example if the deficiency in an area causes concern such the best fit score seems inappropriate then the trainee should score as they see fit but be explicit about the reasons for this.

The scoring matrices would generate 2 grades, one grading the ‘Safe working Environment’ and another for the ‘Effective Educational Environment’. A third score will be available to the quality panel that represents data that the quality team have received from other sources such as the GMC NTS, intelligence from other panels etc. This will be in the same format of Excellent, Good, Requires Improvement, Inadequate and the best fit method. In order to generate an outcome for the post the Quality Panel should note all 3 scores and generate an overall score for the post. Actions that the LEP, department can alter to improve the score should be an explicit outcome of the Quality Panel. This should raise the Quality of Training and ensures we aspiring to excellence.

When considering all three contributing grades from ‘Safe working environment’, ‘Effective Educational Environment’ and Additional Quality data, there are 3 possible combinations:

  1. All three are the same grade in which case and becomes the overall grade
  2. There are 2 of one grade and one of another the majority grade becomes the overall score
  3. Each of the 3 grades are different in which case the middle grade becomes the overall

Anomaly score

On rare occasions one issue may stand out as a problem in a post with otherwise a good score. It is important to capture the problem in the report and so it is proposed to include an ‘Anomaly score’ on the report eg a post may generate on overall ‘Good’ but trainees have no dedicated teaching in which case the report would record the score as Overall ‘Good’ but ‘Requires Improvement ’ for teaching

The scores can be completed before and discussed at the quality panel. It is not intended that each post is meticulously scored at the panel itself. 

Panel Report

Each quality panel will produce a report in a structured format Appendix 3. There is no requirement to document the detailed discussion that has led to the panel’s decision on each post.

Each post (or grouping where appropriate) will be graded as Excellent, Good, Requires Improvement or Inadequate  according to the process described above.

The report must state clearly the action that is required for the post graded to move to the next level i.e Inadequate to Requires Improvement, Requires Improvement to Good and Good to Excellent.

 The report, when agreed by the panel, will be sent to the Education Leads of the posts cited in the report for comment and response. The chair of the panel TPD will respond to any feedback and decide whether this alters the outcome the panel made before forwarding a final Quality Panel report to the Head of School and Quality Team. The Quality Team will then keep a register of the grading of each of the posts and determine how those posts rated as Requiring Improvement or Inadequate and dealt with in the context of the Quality Register and Dean’s Report to the GMC.

Outcome of Quality Panel Report

The report will be submitted to the Head of School and Quality Team for approval and clarification (if required). Once the report is approved it will inform the Quality Register. Those items which meet our thresholds contained within the Quality Framework will be submitted to the GMC via the annual Dean’s Report. The report will also be distributed to the Education Providers hosting the posts via the relevant DME. It is expected that the DME will distribute the information to the departments that host the posts. The report will be published on our website in the Quality Management section.

Those posts that have been graded as Inadequate will be the subject of enhanced management within postgraduate medicine. Those posts graded as Excellent will be included in the postgraduate medicine good practice report.

Quality Panel Reporting Process

 

Quality Panel Matrices

Quality Panel Outcome Reporting Matrix - Peninsula PGME