Interdisciplinary Nursing and Medical Student Teamwork: Results of Knowledge, Skills, and Attitudes Instrument Testing

David Hollar
David_Hollar@med.unc.edu
UNC-Chapel Hill School of Medicine
Gwen Sherwood
gwensher@email.unc.edu
Mary Holtschneider
mary.holtschneider@duke.edu
Karen S. Frush
frush002@mc.duke.edu
Susan Sawning
sawning@med.unc.edu

Objectives: The purpose of this presentation is to describe a new interdisciplinary, interinstitutional health professions teamwork activity and the development and testing of TeamSTEPPS teamwork knowledge, skills, and attitudes (KSA) instruments.

Methods: Four evaluation instruments were developed: (a) a twelve item teamwork knowledge test (several versions), (b) a 36-item teamwork attitudes instrument, and (c) a 10-item standardized patient (SP) evaluation of four-student teamwork skills performance, and (d) a 10-item modification of "always rate" items from Malec et al. (2007, Sim Healthcare 2:4-10) Mayo High Performance Teamwork Scale (HPTS).

Data Source: N=203 senior nursing students and n = 235 fourth-year medical students from two major universities participated in a one-day interdisciplinary teamwork conference, beginning with a didactic lecture, then semi-random assignment of students to four instructional delivery methods.

Results: The pre- and posttest CHIRP attitudes instrument continued to demonstrate strong validity with internal consistencies of .824 and .859, respectively, and an eight factor structure that explained 54.5% of variance in student scores. On attitudes, all four cohorts significantly improved from pre-to-posttest (F1,370 = 48.7, p = .000), but there was no significant difference between cohorts (F3,370 = .325, p = .808). Similar results were found for student knowledge, which significantly improved for all four cohorts, although again there were no significant differences between cohorts.

Conclusions: The teamwork training significantly improved student knowledge, attitudes, and skills concerning teamwork in improving patient safety, although we could not discern between the effectiveness of each educational intervention.

Objectives:
Strategies to improve patient safety include both technological advances as well as non-technical communication and teamwork skills.  IOM reports (1999, 2003, 2004) of the critical impact of teamwork on health care outcomes led to a proliferation of healthcare team training programs. Few have an evidence based approach to identify content, effective pedagogies, and impact of silo education. TeamSTEPPS was developed by healthcare providers and experts in aviation and crew resource management, and evaluated in military and non-military sites and healthcare settings as an interdisciplinary team training program for health care professionals.
Theoretical Framework:
Several studies have documented the importance of communication, interpersonal skills competencies, and simulated teamwork training for improved patient safety in the medical arena, including emergency medicine. Healthcare professionals, especially nurses and physicians, must collaborate, use evidence-based practice, and work in effective teams to promote quality and safety in the provision of exemplary patient care. With the growing national nursing shortage in conjunction with the demand for an improvement in patient care outcomes and a safer healthcare system, there is a need for a new model of nursing and medical education practice that focuses on teamwork, quality improvement, interdisciplinary team care, patient centered care, and evidence based practice and informatics utilization. The authors have extensive experience in promoting interdisciplinary teamwork training via case conferences and emergency preparedness activities. Here, a new interdisciplinary, interinstitutional activity is described with respect to the development and testing of TeamSTEPPS teamwork knowledge, skills, and attitudes (KSA) instruments.
Methods:
Four evaluation instruments were developed: (a) a twelve item teamwork knowledge test (several versions), (b) a 36-item teamwork attitudes instrument, and (c) a 10-item standardized patient (SP) evaluation of four-student teamwork skills performance, and (d) a 10-item modification of "always rate" items from Malec et al. (2007, Sim Healthcare 2:4-10) Mayo High Performance Teamwork Scale (HPTS). The knowledge, attitudes, and SP skills evaluation instruments are original tools developed and validated by the authors. The attitudes instrument was previously validated by the authors with a unidimensional internal consistency for all 36 items of 0.850; maximum likelihood factor analysis yielded nine factors explaining 59.8% of the scale variance (R2 = 0.5977). Statistical analyses of results included GLM Analyses of Variance, reliability and maximum likelihood factor analyses, Independent Samples t-test, and Mann-Whitney U rank test using SPSS 15.0 and SAS 9.1.
Data Source:
        N=203 senior nursing students and n = 235 fourth-year medical students from two major universities participated in a one-day interdisciplinary teamwork conference, beginning with a didactic lecture, then semi-random assignment of students to four instructional delivery methods: (a) simulation (n = 80, equally divided by discipline in groups of 4 students); (b) role-play (n = 80, equally divided by discipline in groups of 4 students); (c) Audience Response System Lecture (n = 140 single large group); and (d) Traditional Didactic Lecture (n = 139 single large group). After lunch, students were rearranged into new groups within their original instructional delivery cohort, and the new groups of four students (two nursing, two medicine) were evaluated on their teamwork performance with a standardized patient (SP). The knowledge and attitudes instruments were given pre- and posttest at the beginning and end of the day. The SP evaluation occurred at the end of the afternoon SP session. 88 SP sessions were video recorded with B-Line Medical LLC (Silver Spring, MD) and rated with the Mayo HPTS by eight trained clinician and non-clinician raters.
Results and Conclusions:
The pre- and posttest CHIRP attitudes instrument continued to demonstrate strong validity with internal consistencies of .824 and .859, respectively, and an eight factor structure that explained 54.5% of variance in student scores. On attitudes, all four cohorts significantly improved from pre-to-posttest (F1,370 = 48.7, p = .000), but there was no significant difference between cohorts (F3,370 = .325, p = .808). There were no significant differences between small and large groups (F1,372 = .068, p = .794), between the simulation and role-play cohorts . (F1,370 = .779, p = .379), or audience response system (ARS) versus lecture cohorts (F1,244 = .273, p = .602)
Similar results were found for student knowledge, which significantly improved for all four cohorts, although again there were no significant differences between cohorts. Students’ scores on the knowledge post-test were significantly higher than their pre-test scores (10.48 vs. 9.22, p<0.0001). Cohort effect was not significant, i.e., scores were not different for the different cohorts. The cohort-time interaction approached significance (p=0.108). Specifically, there was a trend of post-test scores in the simulation cohort being higher than in the role play cohort (10.63 vs. 10.29, p=0.24) and of ARS post-test scores being higher than both role play (10.62 vs. 10.29, p=0.14) and lecture (10.62 vs. 10.38, p=0.162) post-test scores. That is, students in the simulation and ARS cohorts showed a slightly greater improvement in their teamwork knowledge following training than did students in the role play and lecture cohorts.
For the SP evaluation of teamwork skills, there was weak internal consistency between items (.469), although the instrument was not intended to be unidimensional, which was further demonstrated by an oblique maximum likelihood factor analysis. Clinician and non-clinician interrater reliabilities of 9 of 10 SP team skills items was excellent, ranging from .683-.968. Nevertheless, there were no significant differences between cohorts in SP ratings of teamwork skills for the entire 10-item scale or for non-parametric item-by-item comparisons. For the Mayo HPTS, interrater reliabilities among the eight raters were generally poor, ranging from -.665 to +.670 for the 20 SP sessions evaluated so far. The Mayo instrument is currently being revised and a new set of raters are being trained to evaluate the remaining 68 teamwork skills SP videotape sessions.
The teamwork training significantly improved student knowledge, attitudes, and skills concerning teamwork in improving patient safety, although we could not discern between the effectiveness of each educational intervention. The SP scale items had poor internal consistency (reliability), which is fine given that the scale was not necessarily intended to measure one dimension of team performance (see GSK Results). The teamwork knowledge and attitudes instruments demonstrate strong internal consistencies and multifactorial structures that explain close to 60% of variance in student scores while addressing various learning components for teamwork and patient safety. There is excellent interrater agreement on the SP scale (see GSK Results), but there was poor agreement on the Mayo instrument. It is interesting that Item 4 on the Mayo instrument performed best given that it was a spin-off of Item 3 in the original Mayo Scale. It is probable that the Mayo scale is too general and, therefore, is subject to wider variations in interpretation. These four instruments and a new set of complementary evaluation tools are being prepared for a 2008 multi-day interdisciplinary teamwork training exercise involving n = 450 nursing and medical students from the same two major universities.