There is increasingly strong evidence that failures in teamwork and communication in healthcare contribute significantly to unintended treatment injury and poor outcomes for patients.1–3 The operating room (OR) is a high acuity, complex environment in which good communication and teamwork are essential. However, staffing patterns in the OR may predispose teams to errors in communication. The composition of the OR team varies from day to day or even over the course of any one day. There may be limited time for staff to gain an understanding of one anothers’ capabilities and establish the sense of mutual respect and trust required for open communication and effective teamwork. Opportunities for OR teams to share information about the patients, plan for the day and identify concerns may be limited by the way work is organised. OR teams are typically composed of three sub-teams (surgical, anaesthesia and nursing disciplines) with consequent differences in professional backgrounds and clinical responsibilities. These differences and established hierarchies or professional boundaries may inhibit speaking up or sharing of information,4 or lead to variable engagement with safety interventions such as the WHO Surgical Safety Checklist.5
Patient safety has advanced through the study of human error and the identification of defences to prevent repeat failures in the delivery of care6 but this approach tends to be retrospective and reactive. It is equally important to understand why things go right and thereby build resilience into systems, where team members constantly predict future developments, adapt to changing circumstances and enhance success in the management of patients’ problems.7 The fundamental underpinnings of such adaptable, effective teams have been described by Salas8 on the basis of empirical evidence from teams across multiple industries. These underpinning mechanisms are shared mental models, mutual trust and respect, and clear communication. The evidence on surgical team training improving patient outcomes is mounting,9,10 justifying the investment of time and resources.
We have previously described the Multidisciplinary Operating Room Simulation intervention (MORSim).11 MORSim addresses the fundamental underpinnings of effective teamwork described by Salas by bringing whole OR teams together for a day of challenging simulated clinical cases, debriefing and intense discussion. MORSim cases involve simultaneous simulation of nursing, anaesthetic and surgical activities around shared clinical scenarios, with each group dealing with realistic challenges involving their own specialty as well as requiring interaction with the whole team. The aim of MORSim is to improve information sharing in clinical practice, convince participants of the value of effective teamwork and communication through shared simulated experiences, teach specific communication strategies to develop shared mental models and build relationships between OR staff. The ultimate goal of MORSim is to reduce treatment injury and improve outcomes for patients. An important and relatively novel feature of MORSim is the integration of surgical and anaesthesia simulators to enable realistic and challenging clinical tasks to be undertaken by all participants, including open surgical procedures and haemorrhage control. This contrasts with many simulation-based educational endeavours aiming to improve teamwork in the OR, which are either lower in fidelity or lacking in multidisciplinary participation.
We ran 10 MORSim study days for staff from each of the two sites. One anaesthetist (a specialist or senior trainee), two nurses, one anaesthetic technician, one specialist surgeon and one surgical trainee attended each study day. In total there were 120 participants over 20 study days, comprising 20 specialist anaesthetists or senior anaesthetic trainees, 20 anaesthetic technicians, 40 nurses, 20 specialist surgeons and 20 surgical trainees, split evenly between the two hospitals. End-of-day evaluation of MORSim found positive participant reactions to the intervention, self-reported evidence of learning, improved scores for teamwork and communication, and demonstrated proof of concept, feasibility and value of the intervention.11 In-theatre observations before and after MORSim provided some evidence of improvement in observed teamwork scores.12
The Accident Compensation Corporation (ACC) is a New Zealand Crown entity responsible for administering the country’s universal no-fault accidental injury scheme. The scheme provides financial compensation and support to those who have suffered personal injuries, including treatment injuries from surgery. The ACC has provided funding for MORSim to be implemented across all District Health Boards in New Zealand over the next five years.
The aim of this study was to identify factors that would inform implementation and outcomes assessment of the national programme. To this end, we identified the following main questions:
- How did MORSim participants subsequently implement changes in clinical practice?
- What challenges arose while attempting to implement these changes?
- What did participants perceive to be the requirements for change?
To address these questions, we undertook a qualitative study to interview previous MORSim participants on how they incorporated the lessons from MORSim into their clinical work place, using the “Process evaluation on quality improvement interventions” framework described by Hulscher et al.13
Ethics approval was obtained from the Auckland Regional Ethics Committee (NTX/12/EXP/067) and the ethics committees of the two hospitals involved in the study.
We invited a sample of MORSim attendees to participate in a semi-structured interview three to six months after they attended the MORSim intervention. The interview guide is provided in Appendix 1. Email invitations were sent to a randomly selected sample of anaesthetists (specialists and final year trainees) and specialist surgeons from each site, and we interviewed those who responded on a first available basis. We used convenience sampling of available nurses and anaesthetic technicians at the two sites on days identified by the local theatre co-ordinators. We did not invite surgical trainees for an interview. We planned to recruit up to 48 participants or until we reached a point of thematic saturation of the data where no new ideas were being generated in the interviews that would change the identified themes or their description.
Interviews were conducted, transcribed and analysed by one researcher (AC). A second researcher (JW) read through the transcripts and analysis to crosscheck and further refine the themes. These themes were checked for validity by another researcher (JW). The analysis followed a general inductive approach as described by Thomas.14
In total, 48 interviews were conducted with 11 of the 20 specialist anaesthetists or final year trainees (A), 10 anaesthetic technicians of the 20 (T), 20 of the 40 nurses (N) and 7 of the 20 specialist surgeons (S) who had attended MORSim. The split between sites was even for nurses and technicians. There were five anaesthetists and three surgeons from one hospital, and six anaesthetists and four surgeons from the second hospital. Interviews were on average of approximately 30 minutes duration. The following themes were identified: changes in clinical practice, including information sharing strategies; observed changes in others’ work styles; effect on patient management/outcome; sharing learning with staff members and barriers to change in clinical practice.
Changes in clinical practice and lessons learnt
Thirty-five (out of 48=73%) interviewees (A=9, T=9, N=14, S=3) reported at least one positive change in practice. Themes of positive change included improved communication and information sharing, improved confidence, greater awareness of team members and the working environment as well as development of new skills.
Nine interviewees (19%) reported no change in practice resulting from the intervention (A=2, T=1, N=4, S=2). The main reasons given were: no clinical opportunities to try out the new learning; all processes working well already or difficulties translating learning to everyday practice. The majority of participants (n=45, 94%), even some who reported no changes in practice, reported that they learned something new from MORSim, such as the importance of and strategies for communication, the importance of teamwork and planning, and that the intervention provided an opportunity to reinforce existing skills. Some interviewees said they had learned about the importance of taking a pause to plan before engaging in challenging clinical situations. Most reported learning about other team members’ roles, competencies or times of stress.
“In terms of the Checklist, I’ve changed my attitude … saying or highlighting things that are important or that might go wrong or change … and definitely paying more attention … it’s an important time to discuss things.” (A2)
“I’m able to communicate more. Like, if I feel like the patient is at risk in theatre, I’ll be able to say—‘Oh, he might get a pressure area there.’” (N3)
“… just to make sure that everybody got the chance to share information they knew… a couple of the general surgeons are trying to do it here … it feels quite useful … get a sense of the expectations for the day … you can pre-plan your day.” (T3)
“Globally my awareness of ... the needs, or the things I can do to help other people on the team has probably improved.” (S2)
Eleven interviewees referred specifically to using new communication strategies (A=5, T=1, N=5) while 10 reported no change from previous practice (A=2, T=2, N=2, S=4). These strategies included the use of ‘SNAPPI’, a framework for structured call out12 taught during MORSim (Stop, Notify the team, provide your Assessment of the situation, suggest Priorities and Plan, Invite ideas). Interviewees also reported increased information sharing such as verbalising the procedure, speaking up to address concerns about a patient, better handover and pre-case briefing and using a common language among team members. Interviewees reported learning about the importance of sharing information to achieve a shared mental model within their team. Many said they had learned to speak up more confidently, be more explicit or clearer and more directed with their communication. A few suggested that the intervention reinforced what they already knew about communication and teamwork.
“It’s something that simulation has taught me … to try and verbalise what you’re thinking so that other people can pick up on the cues or pick up on what help you need.” (A4)
Observed changes in others’ work styles
Most interviewees (n=31, 65%) talking to this theme had not observed any changes in others’ work practices (A=9, T=6, N=13, S=3). However, 16 interviewees (A=4, T=4, N=7, S=1) commented on positive changes including increased rapport across the team and increased communication (listening/feedback/information sharing).
“… when I used to work with the surgeon before, they didn’t really participate in the checklist or time out—but now on their own they try to explain what they are doing or just their plan, which they didn’t do before. So they do their part now which is quite good.” (N10)
“… the nurse, we developed a relationship that we can follow through from that day, which was great.”(TC2)
Effect on patient management
Twelve interviewees (25%) considered patient management had improved (A=1, T=2, N=9) through improved processes. These included: better communication (better preparation and planning which reduced operative time, prevented mistakes and smoothed processes); more patient-centred care; increased vigilance; or change in patient safety procedure. Reasons volunteered for lack of effect on patient management included the need for a critical mass of MORSim participants or attendees interpreting the training as relevant only to crisis situations.
“If we can share the information beforehand, we can make sure it’s not a surprise and we can get things prepared so that things run more smoothly for the patient.” (T2)
“I’m more aware of the fact that when I’m not 100% sure of what’s going on or where the surgery is heading or exactly where they’re at, I’m just a bit more inclined to say ‘what are we doing now, where are we up to and what’s going on?” (AC2)
Sharing learning with staff members
Thirty-one interviewees (65%) reported positively on sharing learning with other staff (A=6, T=8, N=15, S=2). These reports included incorporating key messages into clinical teaching, orientation of new staff and sharing ideas with colleagues. Interviewees also reported positively on the intervention to their colleagues and reflected on their experiences with others who had attended MORSim. There were examples of role modelling new behaviours and encouraging or empowering others to communicate. Fourteen reported not sharing learning from MORSim with others.
“… talking about the importance of … (a pre-brief) with surgeons that I work with regularly. There’s a list that I do regularly where we seem to just very frequently run into problems with equipment or positioning or things like that so I have talked about that a little bit about—that if there’s some way we can improve the communications before the list so we can try and prevent all these distractions from happening.” (AC5)
“When I was training new staff I was using skills that I learned from the study and kind of talk through and encourage them to be more proactive and get more involved in the teamwork.” (N6)
Barriers to change in clinical practice
Thirty-nine interviewees (81%) reported one or more barriers to change in practice (A=11, T=7, N=15, S=6). Eight reported no barriers (A=1, T=2, N=4, S=1). Many interviewees suggested that time pressures and logistics of how the work was organised made it hard to get all team members in one place at the same time to carry out prompted pauses (eg pre-surgery briefings). Others suggested that some staff at their hospitals could not see the value in concepts of teamwork and information sharing, and there was a culture of not talking about such things or socialising together. Others expressed the view that spending time on pre-surgery briefings or time-out was not considered important or useful for patient management.
The cultural and language backgrounds of some staff were also cited as barriers to the implementation of knowledge learned from the intervention. Some interviewees argued that it was unrealistic to expect staff from some cultures to have the confidence to articulate perceived problems to more senior members of staff. In fact, status hierarchies were more generally identified as a barrier to putting knowledge into practice, regardless of ethnicity.
Limited exposure of staff to the MORSim intervention was seen as a barrier to successful implementation. Sometimes there was only one person in theatre who had participated in MORSim. Some interviewees also stated that frequent changes in team structure made it difficult to develop a culture of teamwork and communication.
“Making sure that everyone stops what they’re doing and participates rather than counting and doing what they’re doing carrying on in the background … we all think we can multi-task but obviously we can’t.” (SC1)
“In a work situation … We don’t stop, we don’t talk about things, we don’t talk about things before or after … The barriers to implementing what I learned are the social barriers that exist in the work environment … we don’t socialise together.” (A1)
“The study there was good but the thing is the attitudes always come from the top of the team. So all the junior ones are wanting to change but if the boss is not doing what he should be doing it is really hard. I guess if we start training all the junior ones then hopefully this culture can carry on and things will get better.” (6N)
This study extends our prior work11 by showing that at least some of the positive changes in attitude and behaviour produced by MORSim, and previously demonstrated in simulated cases over the course of the MORSim training day, are reportedly maintained over time and lead to changes in clinical practice. Neily et al,10 who based their intervention on the principles of crew resource management,15 exposed teams to day-long training involving lectures, group interactions and videos. They were able to show reductions in mortality and improvement in communication and other teamwork attributes in Veterans Administration hospitals in the US. Simulation-based team training is emerging as a popular approach to team training, however, in a recent systematic review of what works in OR teamwork training,13 we found only one study reporting an effect. This was in the form of participant self-report of changes in the OR based on responses of 12 interviewees of whom 50% reported changes.16 A subsequent report on an insurer-funded multidisciplinary simulation-based OR team training intervention also reported that interviewees intended to make changes in clinical practice after the intervention.17 Our study (in a different country and context) adds to the body of literature on simulation-based team training interventions. It goes beyond reporting participant intent and identifies participant reports of actual changes in practice, as well as providing insights into factors facilitating or impeding change.
It is worth summarising some or the key themes that emerged from our study, using the “Process evaluation on quality improvement interventions” framework described by Hulscher et al.13
How did the target group experience the intervention and the changes?
Interviewees described adopting into practice many of the elements of the MORSim intervention and reported positive experiences of change in communication, culture and collaboration. Some interviewees made attempts to spread the MORSim concepts to their colleagues. Many examples were linked to the WHO Surgical Safety Checklist.
What problems arose while implementing the changes?
Time emerged as a frequent impediment to change—participants described a task-focused approach to getting the job done without time to stop for a brief or stop to share information and with limited value placed on opportunities to socialise.
Other factors limiting uptake included a lack of a culture of teamwork and collaboration, lack of leadership and hierarchical relationships. The latter may be more prominent in some ethnic groups.
What requirements for change were identified?
Making time for team-building and scheduling times for communications required commitment from hospital management and clinical colleagues. Importantly, insufficient exposure of staff to MORSim was considered a major short-coming. Staff education needs to include knowledge of evidence on teamwork, error and patient safety, and competence in effective teamwork and communication behaviours. MORSim appears not to have had as powerful an effect on surgeons as it did on the other OR groups. For example, compared to other groups, there were generally fewer positive responses from specialist surgeons in regards to both personal changes and observed change in the OR. This may reflect a hierarchical surgical culture where communication is viewed as a one-way process from surgeons to the rest of the surgical team, or they may overrate their own teamwork.18 This may be a key group to target for engagement in MORSim implementation.
Implications from the pilot study for the national implementation of MORSim
The process evaluation framework described by Hulscher et al.13 suggests the experience of participants in a pilot study can provide important information about the factors associated with success or failure. These interviewees have identified some crucial success and failure factors that will guide our national implementation of MORSim. The interviewees’ perceptions of the MORSim course indicate that it can translate into worthwhile changes in clinical practice. Opportunities for socialising over the course of a full day of MORSim training may be a factor in breaking down hierarchies and professional boundaries and should not be forgotten. Repeated reference by interviewees to the WHO Safe Surgery Checklist identifies this as an opportunity to build on an existing structure, supporting integration of MORSim with existing Health Quality and Safety Commission quality improvement interventions in Checklist administration.19 The question of ethnic differences in perceptions of team hierarchies and speaking up suggests a need to engage with leaders from diverse groups to find the most appropriate intervention.
Resistance to the intervention by interviewees’ colleagues underpins the requirement for a comprehensive, multi-level engagement strategy prior to the national implementation of MORSim. Institutional and clinical leaders need to be convinced that improving teamwork and communication is important for their patients and worth the investment in time. The engagement strategy will need to incorporate evidence that is rigorous, of obvious relevance to patient care and clearly conveyed. A critical mass of OR staff need to be exposed to the intervention at each institution. A strategy where the majority of staff are exposed to MORSim suggests training needs to occur locally, and to be feasible, needs to be run by local staff. This will require building capacity within each DHB to deliver regular MORSim team training, implying the need for a national instructor training programme, provision of resources to run simulation training and ongoing monitoring to maintain the quality of MORSim training.
Strengths and Limitations
The strengths of this study include its focus on change in practice some months after the intervention, and on the relatively high proportion of MORSim participants interviewed (48 of 120 MORSim attendees: 55%, 50%, 50%, 35% of anaesthetists, anaesthetic technicians, nurses, surgeons respectively). It is possible that interviewees may have differed in some systematic way from other MORSim participants despite our efforts to avoid this. Furthermore, interviewees may provide answers supporting their own teamwork and communication skills, or the interviewer may have introduced bias through the framing of questions. However, the interview questions were guided by a schedule and interviewees were encouraged to give honest feedback on the programme, which would remain confidential and anonymous. We therefore hoped to reduce any response bias.
The study was limited to two large city hospitals and the extent to which our findings would predict the experiences of staff from other hospitals remains to be tested. Demonstrating improved patient outcomes was beyond the scope of this study but is an area for future research. As indicated, planned future work involves a national implementation of MORSim, funded through the ACC. This will allow evaluation of the MORSim team training over a range of hospital sizes and surgical disciplines, and has the potential to show reductions in mortality and perioperative harm at a national level.
MORSim had lasting effects on reported attitudes and behaviours in clinical practice. These effects are consistent with more effective teamwork and communication. This study adds to the accumulating body of evidence on the value of simulation-based team training and provides some additional recommendations on how widespread implementation of regular team training for healthcare teams could be accomplished. Time and resources are needed for building, maintaining and enabling behaviours and processes that support effective communication and sharing of information in OR teams.
Ethics approval was obtained from the Auckland Regional Ethics Committee (NTX/12/EXP/067) and the ethics committees of the two hospitals involved in the study.