Admission and poor performance of trainees in the postgraduate GP training in the Netherlands

Abstract

Until 2014 the selection for the Dutch postgraduate GP training was conducted locally, on the eight GP departments. The procedure consisted of a letter of application and a semi-structured interview. We investigated to what extend department of choice, candidates’ characteristics and qualities explained admission into GP training in a nationwide observational study of all candidates who applied in 2009/2010 (n=597). The study population addressed 542 candidates. Sixty-three candidates were rejected on letter of application. So 479 candidates were admitted to the interview, of which 340 were admitted to the GP training. Male candidates and candidates who followed medical school outside north western Europe had more risk of being rejected on application letter. Department of choice had a strong association with admission in both stages while candidates’ qualities explained admission as well. In an observational study of all candidates who entered the Utrecht selection procedure between April 2008 and 2010 (n=394), we investigated the inter-rater reliability of the interview. Twenty-six candidates (4,4%) were rejected on application letter. Ultimately, 206 of the 365 candidates (56,4%) were admitted to the GP training. The inter-rater reliability (Intraclass correlation coefficient (ICC): 0.78 - 0.84) was satisfactory. Reduction from three to two assessors slightly reduced reliability. Candidates’ qualities independently explained admission to the GP training in contrast to individual characteristics. Next, the content of a new competency based selection procedure was determined with a modified Delphi procedure. Consensus on the following CanMEDS ‘ roles was reached: ‘medical expert’, ‘communicator’, ‘collaborator, ‘manager’, and ‘professional’. Instruments were selected by searching the literature for relevant, feasible, reliable and valid methods. Four instruments were included: the National GP Knowledge Test (LHK); a Situational Judgement Test (SJT); a patterned behaviour descriptive interview (PBDI), and a series of three work-related simulations (SIM). In a pilot study reliability and validity aspects of the new procedure were explored. Content-related competencies showed low correlations with one another when measured with different instruments, whereas more diverse competencies measured by a single instrument showed moderate to strong correlations. LHK and SJT were easy to implement, but had poor reliability (ICC: 0.59; 0.55), while PBDI and SIM showed acceptable levels of reliability (ICC: 0.79; 0.73), but were more challenging in implementation. The candidates perceived the new procedure as more fair. Lastly, in an observational cohort study of trainees who started the GP training in Utrecht between 2005 and 2007 (n=215), we investigated the frequency, nature and risk factors of poor performance and attrition among trainees. The overall percentage of poor performance during the training was 22.8%. In the 1st and 2nd years, problem areas among poor performers were equally distributed across the roles ‘medical expert’, ‘communicator’ and ‘professional’. In the 3rd year, a shortcoming in ‘professionalism’ was the most common problem. Higher age was a risk factor for poor performance. Trainees with early sufficient assessment scores in ‘communication’ and knowledge were at lower risk of poor performance. Poor performance in the previous year was a risk factor for poor performance in the 2nd and 3rd years.