“It has been said that democracy is the worst form of government—except for all the others that have been tried” (Winston Churchill). The same can be said for the selection process into medical school.
Most university professional degrees require some form of selection but none experience the same level of scrutiny and hand-wringing as do our country’s two medical programmes. Of course medicine is a prestigious career and quite naturally it attracts large numbers of eager students convinced that they have the right stuff to become good doctors. A large number of them genuinely do have the right stuff, but sadly only 10% ever get the opportunity, leaving the other 90% disheartened and considering plan B.
How we identify that 10% and the qualities we look for in our medical students has long been the subject of hot debate. Two articles in this issue of NZMJ further examine our current process, focusing on two semi-quantitative approaches that are applied to assess personal attributes and cognitive and non-cognitive skills.
In the first paper, Poole et al1 reports on the findings of a working party commissioned to examine the selection interview at the University of Auckland which constitutes 25% of the final grade.
In a second article, Dhar et al2 reports on the student perception of the Undergraduate Medicine and Health Sciences Admission Test (UMAT) which constitutes 15% at Auckland and 33% of the final admission grade at Otago.
The first article is interesting as a point of differentiation between Auckland and Otago while the second article provides a first-time perspective of how students view the standardised UMAT test—the answer is not well!
First off, let’s be clear that both medical schools still use the grade point average (GPA) as the principal selection tool, constituting 60% of the final grade at Auckland and 66% at Otago. At Auckland the scores from UMAT and the interview only apply to a sub-group of students who exceed a high GPA score. At this stage the students know they have a 50% chance of selection. So what’s the added value of the interview and UMAT and are they both fair? Although the former question is partially answered, the later remains debatable even though the students themselves say that UMAT is not fair.
The first Dean of the Auckland School of Medicine Cecil Lewis put it bluntly: the interview is to “identify the bad buggers”. Although choosing less colorful prose, I agree that the interview still attempts to do this but admittedly not that successfully—but it does achieve more.
The Auckland Working Group commissioned by my predecessor Prof Iain Martin was charged with determining the value of the interview and recommending whether (a) it should continue and (b) if so, in what form?
The Working Group admits that the interview for the most part has variable reliability but still recommends retention, citing multiple positive aspects. First it provides the student with an opportunity to “sell themselves” at the end of a relatively brutal and impersonal first year.
Even unsuccessful students strongly support the interview as an opportunity for personal engagement. Each student goes into the interview knowing that there is a “must-have” category where selection is guaranteed for those with exceptional personal qualities (notwithstanding the fact that they have a high GPA). Even though only a handful of interviewees achieve this category, and likewise even fewer are deemed to be at the other end of the scale, this remains a positive incentive for students.
The vast majority score in the three middle categories so for them the comparative effect of the interview is minimal. It does provide, as Poole et al highlights, a point of distinction for entry into Auckland Medical School, a sense of involvement by the student, a sense of involvement by the community as interviewers and a sense that selection is not just about exam performance.
The biggest reason for not continuing with the interview is cost. With rising student numbers, it is becoming a logistical headache to organise 450-plus interviews each year. One seriously considered alternative was choosing a larger group based on GPA and then selecting by lottery. This meets both the excellence and the fairness criteria, but seems contrary to the goals of a school dedicated to selecting students of good character and maturity. Under this scheme, the “must-haves” would have much less of a chance. Auckland will continue with the interview for the foreseeable future.
The UMAT is a different beast. A standardised test organised by the Australian Council of Education Research, it provides a benchmarked assessment for all medical students across Australasia. Despite its widespread use, there is very little data to show that it is an effective method for ranking students, particularly in the area of non-cognitive social skills.
In the article by Dhar et al the simple question is asked—what do the medical students think about UMAT? The answer appears to be a resounding thumbs-down. The students consider the test to be high cost and stressful. More worrying is that commonplace rehearsal (at considerable cost to the student) does increase success. Those students who cannot afford to rehearse are immediately disadvantaged. In the study, 21% of medical students reported sitting the test more than once and 70% indicated that they got higher marks in the subsequent test.
It is not surprising therefore that a profitable cottage industry has sprung up where current medical students offer to train new students in UMAT for a fee. Entrepreneurial it may be, but strongly discouraged and difficult to stop. Clearly UMAT has encouraged undesirable behavior. To counter this, students can now only sit UMAT once and it must be in the year of application to medical school.
These two studies highlight the importance of ongoing review and being able to justify the use of more than one assessment tool for selection. Like democracy, no selection system is perfect but it is important to ensure that the system chosen meets the needs of the programme and is fair and equitable to all students.