1st December 2017, Volume 130 Number 1466

Michelle R Wise, Bridget Kool, Lynn Sadler, Roshini Peiris-John, Gillian Robb, Susan Wells

Clinician engagement in ongoing health service quality improvement (QI) is an important component of improving clinical outcomes. Integrating QI in undergraduate and postgraduate medical education to equip clinicians with the knowledge and skills to improve care is increasingly acknowledged as important. A number of international medical accreditation bodies now include QI as a core competency, necessitating medical education programmes to develop authentic curricula in this area.1–4 The New Zealand and Australian Curriculum Framework for Junior Doctors includes clinical audit as a core element of all pre-vocational training.5,6 Like many other professional colleges, clinical audit is also a mandatory requirement set out by the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG).7

A growing number of universities worldwide are offering QI learning opportunities to medical learners, although there is significant heterogeneity in educational content, teaching methods, students targeted and learning outcomes.8 Undergraduate curricula vary from a few lectures (the majority involving fewer than 10 contact hours and often consisting of a single session) to the undertaking of a QI project.9 Postgraduate curricula targeting junior doctors often involve multiple encounters within existing core rotations, standalone sessions or elective rotations with participation in QI projects.8,10 The primary objective in most QI curricula is learning the principles and methods of QI, with common educational content including QI science and skills, safety incident analysis and systems thinking.8 Assessments of QI curricula have reported improvements in learner’s knowledge and self-efficacy, with lesser emphasis on changes in behaviour and clinical process.8 Incorporating experiential learning that is fully integrated into day-to-day clinical work has been suggested as a way to improve generalisability to future practice settings and lead to change in clinical practice.11

To date, the most promising form of experiential learning in undergraduate QI curricula combines classroom learning with practical projects.12 The QI projects commonly include elements of root-cause or safety incident analysis, a chart audit and reflective practice.13 Students are assigned to interprofessional QI teams14,15 or to community settings,16-18 or undertake mentored projects.19 Difficulties in achieving a balance between both didactic and experiential learning exist amidst competing educational demands of medical students. However, the QI project approach is consistent with adult learning principles, where learners are engaged in actual QI practice and the projects have the potential for improving clinical processes of care.13

Typical QI projects require iterative cycles of implementation and evaluation. However, longitudinal learning attachments and integrating QI learning into day-to-day clinical work are often not feasible.11 The competing educational demands on medical students require projects to be kept simple and workload appropriate.20 The aim of this paper was to describe how we incorporate experiential QI learning at the University of Auckland by integrating a clinical audit project into the Year 6 obstetrics and gynaecology clinical attachment.

Methods

At the University of Auckland, we provide classroom and web-based QI science and skills learning during the six-year undergraduate medical curriculum. Since 2004, we have also incorporated experiential QI learning by integrating a clinical audit project into final-year medical students’ obstetrics and gynaecology (O&G) clinical attachment across nine hospitals.

The improvement sciences theory and skills learning commences in Year 3 where the focus is on patient safety. Medical students are allocated into interprofessional teams with third-year nursing, pharmacy and optometry students for a two-day module which consists of face-to-face lectures covering teamwork in healthcare, understanding healthcare disparities, error and violation, human factors (such as practitioner fatigue) and measuring intervention impacts (include revenge effects). The module culminates in conducting a root cause analysis following an adverse event, and the process of open disclosure. Students in Year 4 have six hours of lectures on the critical appraisal of epidemiological evidence. In Year 5, students complete an eight-hour online self-directed module on clinical audit and improvement frameworks, sampling and measurement strategies, and QI science tools to enable them to display, describe and learn from clinical data.

In Year 6, students apply their improvement science skills to conduct a clinical audit project on a topic of their choice during their clinical attachment in O&G. The Year 6 QI programme is designed to help students view a health service through the lens of QI dimensions and the concepts of harm, waste and variation. Starting with the research evidence for best practice, students follow the audit cycle (see Figure 1).

Figure 1: The clinical audit cycle. 

c 

Figure credit: Health Quality and Safety Commission, New Zealand.  

Working with a clinical supervisor, the students identify an area for potential improvement and set an evidence-based standard of care. They conduct part of a simple clinical audit to measure current practice and investigate reasons for any deviation from the standard. Students put into practice basic approaches to measurement in QI, including identifying appropriate measures and sampling methods, collecting, analysing and interpreting data, and describing and displaying variation. Students work alone or in pairs and are allocated one academic half-day per week to complete the project. Students are encouraged to discuss their findings with the relevant stakeholders to inform their interpretation of the findings and to develop a suggested action plan. Although the QIPs are carried out during a short timeframe, Year 6 students do learn QI skills beyond conducting a clinical audit. Students are expected to discuss their findings with the relevant service providers, interpret their findings by describing the potential root causes and how they fit in the context of the service, and provide change ideas and recommendations of possible solutions that the service could implement. They also identify areas for further investigation. The students present to the local clinical team for discussion. We are aware of many service changes that have been made by clinicians as a result of the student presentations over the years. We then encourage future students to re-audit to check if the change/s has resulted in any improvement, thus closing the loop. The projects are formally assessed based on a written report and an oral presentation to their peers, QI faculty and where possible the clinical team.

Results

As at July 2016, over 1,250 projects have been completed, with two publications arising from this work so far incorporating multiple audits from multiple sites,21–22 and a thesis dissertation. The projects cover a broad range of obstetrics and gynaecology topics. They range from being condition-specific (eg, adherence to clinical indications for elective caesarean section before 39 weeks; prophylactic Anti-D administration for Rhesus (D) negative pregnant women) to general processes of care and practice (eg, adherence to handwashing policies; screening for family violence on admission). The topics chosen most often by students over the last 14 years include post-operative thromboprophylaxis (n=39 projects), induction of labour (n=53), colposcopy (n=98) and antenatal screening (for infections, congenital anomalies, gestational diabetes, anaemia, etc.) (n>100). A clinical audit project example is provided in Table 1.

Table 1: Example of a clinical audit project.

Topic selection

 

Preterm birth complicates 7% of New Zealand births, and accounts for 85% of perinatal morbidity and mortality; antenatal corticosteroids reduce adverse outcomes

Standard of care

 

Criterion (antenatal corticosteroids are given to women at risk of preterm birth 24+0–34+6 weeks); target (100%); allowable exceptions (imminent birth)

Survey current practice and compare against standard

Process indicator (receipt of antenatal corticosteroids); study population (every second woman who gave birth from 24+0–34+6 in 2013 at Auckland Hospital, exclusion: if patient declined); sample size (106); variables (type of maternity caregiver, gestational age)

Display and interpret findings

85% of women received antenatal corticosteroids; presented data using a pareto diagram and an Ishikawa diagram

Develop an action plan

Local teaching sessions on clinical guidelines; add clinical guideline to orientation of new staff; link to clinical guideline within electronic medical record; add date/time of antenatal corticosteroid doses to electronic medical record; ensure clear documentation of gestational age 

A formal evaluation of the QI Programme was conducted in 2014 when a shorter clinical rotation in O&G (five weeks to four) was to be implemented. Details of the formal evaluation are reported elsewhere.23 This evaluation was funded by a University of Auckland Learning Enhancement Grant. In summary, the QI programme was seen by clinicians as a valuable way for Year 6 medical students to apply their knowledge and skills into practice, by performing a hands-on project within the constraints and context of a busy women’s health service. Additionally, most clinicians agreed the QIP has provided value to the O&G service and the organisation. Students reported gaining useful insights into QI and agreed that the skills learnt would be important for their future.

However, the evaluation revealed that there was significant variation in data collection and in programme implementation across the eight teaching sites. In addition, challenges were faced by students in selecting a topic that fulfilled project criteria, was relevant to the clinical supervisor and/or the service, and was achievable in the time frame. The findings of the evaluation led to the development and implementation of project-wide and site-specific solutions to ensure learning outcomes continue to be met. This has been largely achieved by developing new resources and making processes more efficient. All resources are now easily accessible to students and supervisors through the university education online portal, such as Year 5 teaching resources, a how-to guide, all previous project reports and a discussion board of ‘tips and hints’ from previous students. A fifteen minute ‘QI Project Overview’ video was developed, highlighting the key components of the clinical audit, the process to follow, timelines and expectations. A ‘topic selection form’ was developed to help ensure students select an appropriate topic and methodology (eg, standard, sample size, variables, etc.) and requires signoff from the supervisor. A ‘written report template’ was created to ensure reports are concise and relevant to stakeholders. Clinical supervisors at each hospital keep an updated list of potential topics, and have created a repository for written reports accessible to hospital clinicians. Hospital QI staff established processes to enable quicker and easier access to medical records.

In advance of the attachment, a welcome email was sent to students providing a link to a university education online portal. The key process steps are summarised below:

  • Week 1: Orientation and topic selection
    • Students and supervisors watch “QI project overview” video, and review available resources.
    • Students receive an encrypted memory stick (to protect patient privacy) for data collection.
    • Students meet with supervisor, identify topic and complete Topic Selection Form.
  • Week 2–3: Conduct project
    • Supervisor signs off Topic Selection Form with specific feedback to students.
    • Students collect and analyse data.
    • Students discuss findings with stakeholders and brainstorm barriers and enablers.
  • Week 4: Complete project
    • Students prepare and submit three-page written report.
    • Students prepare and deliver a 15-minute oral presentation and answer questions.

The QI curriculum meets the University of Auckland graduate learning outcome for students to identify feasible strategies to improve health that incorporate the broader determinants of health at community and population level.24 It also meets the Australasian Medical Council (AMC) accreditation standard regarding QI, which requires students to be able to describe a systems approach to improving the quality and safety of healthcare.3 The 2005 AMC report noted “the team was impressed especially by the root cause analysis of sentinel events undertaken in Year 3 in a multidisciplinary training environment, and the Year 6 quality assurance project.”

We are committed to disseminating what we have learned through the delivery of our QI teaching programme. Recent efforts have included presentations at local faculty medical education rounds, and facilitation of a QI teaching workshop at the 2014 AMEE (An International Association for Medical Education) conference in Milan, Italy, by two of the authors (MRW and BK). We have appointed a part-time research coordinator (RP-J) to assist with resource development, research outputs and collaboration across all sites.

Discussion

Performing a hands-on project within the constraints and context of a busy O&G service is a feasible and effective method of teaching QI and provides value to the health service. Experiential QI learning, as described here, enhances clinical teaching and training. By integrating clinical audit projects within clinical attachments, medical schools can equip future clinicians with the necessary knowledge and skills to incorporate QI into their daily practice and improve clinical practice.

The Institute of Medicine’s ‘Learning Healthcare System’ report described an approach for integrating clinical research and clinical medicine.25 This has subsequently evolved into a broader concept of a learning health system where all stakeholders can securely, effectively and efficiently contribute, share and use data to create knowledge and improve health outcomes. We plan to support a learning network of clinical supervisors and university academics with a view to enhancing QI training, benchmarking service performance across regional hospitals, sharing ideas for improvement and engaging in academic research.

The importance of developing capability of clinical trainers in QI, transforming them into teachers of QI, has been previously highlighted as a way forward in improving QI in medical education.13 The learning network will facilitate this. We are building faculty capacity by involving more clinicians in student projects, and providing support from the academic department and hospital QI staff to guide them in supervising students to learn the required skills. Specifically, when clinicians first become involved in the project, we provide one-on-one tutoring in clinical audit methodology and mentor them with assessment, by co-evaluating student reports and presentations. We then provide ongoing feedback on the topics, helping them appropriately frame the topic as audit (not research). Informal feedback has been that clinicians feel more confident as time goes on to be able to provide students with adequate support and guidance. As they start using the students’ findings to drive implementation of solutions to improve service, they become more aware of the utility of the student projects, and thus motivated to become more involved in future projects. An unexpected finding in our formal evaluation of the QI programme was that clinicians felt they were learning QI skills from the students. This is important as the New Zealand Medical Council requires all clinicians to perform an annual clinical audit of their own medical practice in order to recertify.26

Together with clinician supervisors, we are also working towards documenting changes in clinical processes and patient outcomes that occurred as a result of student projects. Future research could look at whether focus on specific topics could impact the quality of care delivered in O&G, as was shown in improved health outcomes for patients with diabetes at community practices where medical students and their preceptors completed QI projects.16

Although students are encouraged to learn and apply QI principles beyond conducting a clinical audit, we acknowledge that it is challenging to achieve improvements in service and patient outcomes using small projects conducted in short blocks. That being said, we have worked closely with clinicians over the years to direct the student reports to the clinicians who have governance over the areas studied, and to engage with hospital-wide QI departments, in order to facilitate using the findings to improve service. One way to overcome this could be the team-based approach adopted by some postgraduate programmes in which individual junior doctors work on one part of an overall project, building on work done by their predecessors and a quality initiative over time.27,28

Our QI programme is undergoing continuous improvement. We continue to look for ways to modify the clinical audit project in order to reduce student workload. Our supervisors encourage students to choose only one standard, one or two measures, and a few stratifying variables; we are considering assessing the project proposal only, rather than the completed project; and we are exploring with other departments to collaborate with O&G to enable the clinical audit to be conducted over more than one placement, which would extend the project timeframe. We also want to engage clinicians at more sites to improve knowledge translation, for example, getting the findings from student projects back out to stakeholders who can use them to make a difference in their services. Clinicians who see improvements in patient outcomes will be more likely to seek out continuing QI education for themselves and to supervise future students.

Another challenge we are addressing is that of assessment. At the moment individual students are assessed for their QI competency by evaluating their group project presentation. There may be more effective ways of assessing competency and we may add other objective measures of student knowledge such as the QIKAT-R,29 or modify our assessment scheme to a validated tool.30 We plan to survey postgraduate trainees to assess retention of QI knowledge and skills, and investigate whether the projects increased QI capability so that learners were able to improve health service performance in their future roles. The survey will also gauge whether involvement in these projects enhanced exposure to rigorous research methods and structured reporting. As previously reported, this may potentially inspire health professionals to be engaged with academic medicine during their career.31

Medical schools have an integral role to play in ensuring future healthcare professionals are equipped with QI knowledge, skills and attitudes so that they can contribute towards improved clinical practice and patient outcomes. We provide a combination of didactic and experiential QI learning opportunities. The literature notes numerous benefits of experiential learning, although learning real-life QI practices is comparatively new. Incorporating clinical audit projects within short clinical attachments should be considered a feasible and effective method of QI education to medical students. The authors are happy to be contacted to share resources, consult with faculty or facilitate workshops.

Summary

Final year medical students at University of Auckland perform a clinical audit project within the constraints and context of a busy women’s health service. Students work with a clinical supervisor to identify an area for potential improvement, set a standard of care, measure current practice, investigate reasons for not achieving the standard and make real-world suggestions to close the gap between research and observed practice. Since 2004, over 1,250 student projects have been completed, many of which have resulted in actual improvements to clinical care. Such experiential learning during medical school is important in preparing future doctors to incorporate quality improvement knowledge and skills into their daily practice.

Abstract

Aim

To describe how we incorporate experiential quality improvement (QI) learning at the University of Auckland by integrating a clinical audit project into the Year 6 obstetrics and gynaecology clinical attachment.

Method

Students gain insight into the relevance of QI while engaged in day-to-day clinical work. Students work with a clinical supervisor to identify an area for potential improvement, set a standard of care, measure current practice, investigate reasons for deviation from the standard and make real-world suggestions to close the gap between best evidence and observed practice.

Results

Since 2004, over 1,250 projects have been completed, and two journal articles published. Many of the student projects result in actual improvements to clinical processes of care, and lead to strengthening of academic and service provider learning networks and partnerships.

Conclusion

Performing a hands-on project within the constraints and context of a busy women’s health service is a feasible and effective method of teaching QI. Medical schools have an integral role to play in ensuring future healthcare professionals are equipped with QI knowledge, skills and attitudes. Experiential QI learning enhances clinical teaching and training, and is important in preparing future clinicians to incorporate QI into their daily practice.

Author Information

Michelle Wise, Epidemiology and Biostatistics, School of Public Health, FMHS, University of Auckland, Auckland; Bridget Kool, Epidemiology and Biostatistics, School of Public Health, FMHS, University of Auckland, Auckland; Lynn Sadler, Epidemiology and Biostatistics, School of Public Health, FMHS, University of Auckland, Auckland; Roshini Peiris-John, Epidemiology and Biostatistics, School of Public Health, FMHS, University of Auckland, Auckland; Gillian Robb, Epidemiology and Biostatistics, School of Public Health, FMHS, University of Auckland, Auckland; Susan Wells, Epidemiology and Biostatistics, School of Public Health, FMHS, University of Auckland, Auckland.

Acknowledgements

Prof Peter Stone and A/Prof John Buchanan for having the foresight to embed QI teaching in the undergraduate medical curriculum at University of Auckland; Michelle Carvalho for her ongoing administrative support of the medical students in the Department of Obstetrics and Gynaecology; Faith Mahony for her assistance in conducting the 2014 formal evaluation of the QI programme; and Year 6 medical students Prasadi Jayasinghe and Cathy Zhong, on whose 2014 clinical audit project formed the example provided in the paper.

Correspondence

Dr Michelle R Wise, Senior Lecturer, Department of Obstetrics and Gynaecology, Faculty of Medical and Health Sciences, University of Auckland, Private Bag 92019, Auckland 1142.

Correspondence Email

m.wise@auckland.ac.nz

Competing Interests

Dr Wells reports grants from Health Research Council of New Zealand, grants from National Heart Foundation of New Zealand, grants from The Stevenson Foundation, grants from Roche Diagnostics Ltd, outside the submitted work.

References

  1. Accreditation Council for Graduate Medical Education. Common Program Requirements: General Competencies. Chicago, Illinois; 2007.
  2. General Medical Council. Tomorrow’s Doctors London: 2009 [cited 2016 Aug 29]. Available from: http://www.ub.edu/medicina_unitateducaciomedica/documentos/TomorrowsDoctors_2009.pdf
  3. Australian Medical Council. Standards for assessment and accreditation of primary medical programs. Kingston; 2012.
  4. Frank JR, Snell L, Sherbino J, editors. CanMEDS 2015 Physician Competency Framework. Ottawa: Royal College of Physicians and Surgeons of Canada; 2015.
  5. Graham IS, Gleason AJ, Keogh GW, et al. Australian Curriculum Framework for Junior Doctors. Med J Aust. 2007; 186(7 Suppl):S14–9.
  6. Medical Council of New Zealand. New Zealand Curriculum Framework for Prevocational Medical Training. Wellington; 2014 [cited 2016 February 16]. Available from: http://www.mcnz.org.nz/assets/News-and-Publications/NZCF26Feb2014.pdf
  7. Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG). The CPD Curriculum Framework [cited 2017 April 22]. Available from: http://www.ranzcog.edu.au/RANZCOG_SITE/media/RANZCOG-MEDIA/CPD/CPD-Framework-Table-ONLINE-RANZCOG-website.pdf
  8. Wong BM, Etchells EE, Kuper, et al. Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med. 2010; 85(9):1425–39.
  9. Windish DM, Reed DA, Boonyasai RT, et al. Methodological rigor of quality improvement curricula for physician trainees: a systematic review and recommendations for change. Acad Med. 2009; 84(12):1677–92.
  10. Jones AC, Shipman SA, Ogrinc G. Key characteristics of successful quality improvement curricula in physician education: a realist review. BMJ Qual Saf. 2015; 24(1):77–88.
  11. Wong BM. Reporting on Patient Safety and Quality Improvement Education: Designing Projects for Optimal Dissemination. J Grad Med Educ. 2015; 7(4):513–6.
  12. Health Foundation. Evidence scan: quality improvement training for healthcare professionals. London, UK; 2012.
  13. Wong BM, Levinson W, Shojania KG. Quality improvement in medical education: current state and future directions. Med Educ. 2012; 46(1):107–19.
  14. Chessman AW, Bellack JP, Lahoz MR, et al. Students add value to learning organizations: the Medical University of South Carolina experience. Qual Manag Health Care. 1998; 6:38–43.
  15. Hall LW HL, Cox KR, Deane K, et al. Linking health professional learners and health care workers on action-based improvement teams. Qual Manag Health Care. 2009; 18:194–201.
  16. Gould BE, Grey MR, Huntington CG, et al. Improving patient care outcomes by teaching quality improvement to medical students in community-based practices. Acad Med. 2002; 77(10):1011–8.
  17. Knapp M, Bennett NM, Plumb J, Robinson J. Community-based quality improvement for the health professions: balancing benefits for communities and students. J Interprofessional Care. 2000; 14:119–30.
  18. Mak DB, Plant AJ. Reducing unmet needs: a prevocational medical training program in public health medicine and primary health care in remote Australia. Aust J Rural Health. 2005; 13:183–90.
  19. Tartaglia KM, Walker C. Effectiveness of a quality improvement curriculum for medical students. Med Educ Online. 2015; 20.
  20. Gould BE, O’Connell MT, Russell MT, et al. Teaching quality measurement and improvement, cost-effectiveness, and patient satisfaction in undergraduate medical education: the UME-21 experience. Fam Med. 2004; 36:Suppl:S57–62.
  21. Wise M, Sadler L, Ekeroma A. Chlamydia trachomatis screening in pregnancy in New Zealand: translation of national guidelines into practice. J Prim Health Care. 2015; 7(1):65–70.
  22. Lilic N, Jayanatha K, Stone P. Routine postnatal use of the Kleihauer-Betke test for anti-D administration in rhesus D-incompatible pregnancies: a medical student quality improvement project. Jt Comm J Qual Patient Saf. 2013; 39(7):324–7.
  23. Kool B, Wise MR, Peiris-John R, Sadler L, Mahony F, Wells S. Is the delivery of a quality improvement education programme in obstetrics and gynaecology for final year medical students feasible and still effective in a shortened time frame? BMC Med Educ. 2017; 17(1):91.
  24. Faculty of Medical and Health Sciences. Phase 3 Guidebook Auckland: Univeristy of Auckland; 2015 [cited 2016 August 29]. Available from: http://mbchb.auckland.ac.nz
  25. Institute of Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC; 2007.
  26. Medical Council of New Zealand. Recertification and continuing professional development booklet. Wellington; 2014.
  27. Neuspiel D, Hyman D, Lane M. Quality improvement and patient safety in the paediatric ambulatory setting: current knowledge and implications for residency training. Pediatr Clin North Am. 2009; 56:935–51.
  28. Tess AV, Yang JJ, Smith CC, et al. Combining Clinical Microsystems and an Experiential Quality Improvement Curriculum to Improve Residency Education in Internal Medicine. Acad Med. 2009; 84:326–34.
  29. Singh MK, Ogrinc G, Cox KR, et al. The Quality Improvement Knowledge Application Tool Revised (QIKAT-R). Acad Med. 2014; 89(10):1386–91.
  30. Leenstra JL, Beckman TJ, Reed DA, et al. Validation of a method for assessing resident physicians’ quality improvement proposals. J Gen Intern Med. 2007; 22(9):1330–4.
  31. Lawson McLean A, Saunders C, Velu PP, et al. Twelve tips for teachers to encourage student engagement in academic medicine. Med Teach. 2013; 35:549–54.