Dr Ashley Reece
We all remember 2012 – a glorious year of the London Olympics and the Queen’s Diamond jubilee. It was also the year the Royal College of Paediatrics and Child Health (RCPCH) introduced the START assessment. This was the college’s exit exam although it is not at the ‘exit’ of training and is, famously, ‘not an exam’. Until it arrived there was no final check of senior paediatric trainees’ “abilities” before they completed their training and were entered onto the specialist register to be eligible to become a Consultant. So they needed to be assessed in some way to check they were on track to be able to work competently and safely as new Consultants once they had been awarded their CCT (Certificate of Completion of Training). Indeed the GMC (General Medical Council) mandated the RCPCH to do it.
So, in true Paediatric style, a more formative (than summative) assessment was devised, the former often described as an assessment for learning and the later an assessment of learning, where the assessment itself is part of the learning episode or journey.
What is START?
START is an acronym for the Specialty Trainee Assessment of Readiness for Tenure and I add ‘as a Consultant’. It is undertaken usually during the penultimate training year, ie., ST7 year. I like to think of it as an ‘out-of-workplace’, multi-scenario, OSCE-style, formative assessment on standardised scenarios with trained, external assessors.
It began its life as a more OSCE/viva type of assessment and since it was to be done during the ST7 training year, it was known initially as the ST7 Assessment, or ST7A. It has evolved from those early pilots into the format in use today. It is not an ‘exam’ because there is no pass/fail and no grades are given and there is also no resit. It is formative because feedback, both positive and constructive, is given to the trainees. So, really, it is not an exam. However it is mandated as part of the RCPCH’s assessment strategy (https://www.rcpch.ac.uk/education-careers/training/progress/curriculum#assessment-strategy). In order to assess the large number of senior trainees required, an OSCE-style format is used and with a multi-station (each station known as a ‘scenario’) set up, there is a feeling it is like an exam, but it is not. The in-scenario performance is benchmarked against a set, agreed benchmarking standard of a newly appointed Consultant and for each scenario, a global grading of performance is given, not a score. Feedback on how the trainee approached each scenario is given to them through Kaizen (the RCPCH electronic portfolio used by paediatric trainees) some weeks after the assessment. The idea is to discuss this feedback with the trainee’s educational supervisor and devise a Personal Development Plan to work on in the final year of training as the trainee develops towards the transition to being a Consultant.
Transitioning between grades in medicine is hard. There have been many attempts to study movement into the senior grade, some studies using self-reporting of skills and competencies, before the transition compared to after it, and others focusing on readiness for what are called ‘softer skills’ of Consultant working which are not actually ‘soft’, but actually very ‘hard’. They include aspects of management such as service design, planning and writing business cases, people management and aspects of governance.
START was never meant to be a test of knowledge but more so, the application of knowledge. We know knowledge is tacit (implied) and underpins our clinical assessment, decision making and clinical management. However, START is designed more to assess in many of those areas of consultant working which are not about specific clinical aspects of patient management. All the paediatric specialities are included, and the current format is 6 generic scenarios and 6 specialty specific.
So it’s not about how to deal with the difficult asthma; we are not interested in what medications to use, or strategies to support compliance (although that knowledge is important and should be there). What we are interested in are reasons for the poor compliance, how to engender engagement in the treatment and what else might be contributing. Specialist clinical knowledge should be acquired and tested in the workplace. The clinical experience of working in that speciality following the RCPCH Progress curriculum under the Speciality Advisory Group, will ensure appropriate knowledge is gained. In START the speciality provides the context.
Within the scenario there is always some nuance or twist, something to work out. It does not have to be about the patient but could relate to how you deal with a colleague, who may be peer, a more junior doctor or another health care professional. It could be about a service provision, development or change. Then there is a need to assess teaching, prescribing and critical appraisal skills. For the last areas we can try and get right to top of Miller’s pyramid – assessing the actual doing within the assessment.
So if you have not noticed already, we have pretty much covered all of the 11 Progress domains (except ‘procedural skills’ which is left for the clinical arena for obvious reasons).
In almost all the diets we have run so far, we have used the RCGP ‘examination (sic) centre’ in London. This is so we can get around 150 trainees through the assessment over 2 days. With the multi-scenario, OSCE style, with bells ringing and door knocking at set times, and the 4 minutes reading before going in it has the potential to feel like an exam situation. However the big difference is the in-scenario interactions. No ‘mask-like’ face, no laminates to hide behind (looking at you, APLS) no ‘”is there anything else…?” question as time ticks down. The assessors are encouraged to use open questions, facilitate active discussion and to let the trainee talk. We suggest trainees think about the scenario as if they had brought it to a more senior colleague for some sagely advice and to talk it through. The interactions are based on a work by Donald Schön who saw inherent value in having a professional conversation to support and direct learning and understanding.
The biggest difference between the summative assessments and START is the feedback provided to trainees a few weeks after the assessment. We expect this is written to ‘you’ i.e. in the 2nd person, and reflect your in-scenario performance and offer directive feedback on how it went, what was done well and any points to consider or advice about areas to work on in light of this in-scenario performance.
Coronavirus has meant there has been a rethink about how START is delivered. So very recently there has been a bespoke virtual START assessment using . Whether this is a model we can use going forward is something the College’s Education and Training team are considering, especially if face-to face assessments cannot happen at the moment.
A validation and review on feedback received from START since its inception has been recently published showing trainees do value the opportunity to rehearse the thinking behind such tricky areas of Consultant working. Some may perceive it as a hurdle to get over, which in truth it is, as the GMC mandate the college to hold it, and it is necessary for progression into completion of training. But many trainees feel it is a good preparation for a consultant interview although it is should never be used or reviewed at interview, nor should it inform that process. It is one of the pieces of multiple evidence in a portfolio informing ARCP outcome at ST7. In the main it has been positively received and is now embedded. As the person who STARTed it, I am proud of how it has been embedded. It supports the entry into the next phase of your career and is not the exit!
Top tips:
Revising for START is not necessary – your training itself would be enough. But we know as medics we like to prepare – so brush up on your critical appraisal skills, get some of your prescription charts reviewed by the pharmacist and think about your effective teaching skills
- Ask your clinical and educational consultant supervisors what challenging or tricky situations they have had in the last few weeks – many scenario generators bring those real-life situations to work up as scenarios for START.
- Don’t expect to be spoon fed in the assessment; the idea is you have a professional conversation with the assessor – better still if you lead that conversation.
- Use your feedback wisely – and use your educational supervisor; be sure to reflect and create a plan with your supervisor, even if seemed to go well. Your ARCP will go more smoothly with reflection on START than without it.
- Keep an eye on the RCPCH START web pages for latest updates on applications for the next diet and any refinements to the current set up and structure.
Key learning points:
- START is not an exam
- It’s a check on whether you are developing the skills to become a consultant.
- The feedback is to be discussed with your educational supervisor and you create a PDP together to work on in your final year of training.
- It’s novel. Many trainees find it an enjoyable experience.
Further information:
https://www.rcpch.ac.uk/education-practice/start
https://www.rcpch.ac.uk/resources/rcpch-start-guidance-trainees
With thanks for helpful input from the current RCPCH START Chair, Srinivasarao Babarao, Consultant Neonatologist, Wirral University Teaching Hospital NHS Foundation Trust and the Education and Training Committees at the RPCH.
About the author:
Ashley is a consultant Paediatrician and Educationalist at Watford General Hospital. He has co-edited a Paediatric Outpatients textbook. He was the first chair of the RCPCH START Executive and was involved in developing, devising and incepting the first assessment. He is the RCPCH Officer for Assessment and the Director of Medical Education at his Trust.
Twitter: @drashleyreece
about.me/ashleyreece