International Journal of Healthcare Simulation

Introduction

Debriefing in healthcare simulation may be challenging due to factors associated with the learners, the facilitators, the simulation environment and/or other external variables [1]. Learner-related factors that influence the nature of conversations during debriefing include inherent affect, personality, previous experiences or personal biases [2,3]. The degree of learner engagement, interest and emotion influences the process and outcomes of debriefing [4].

Poorly facilitated debriefings may compromise psychological safety and result in missed learning opportunities. Facilitators may use verbal and non-verbal communication strategies (e.g. eye contact, body language) to build and maintain psychological safety during debriefing [4]. Non-verbal communication accounts for a large proportion of the meaning conveyed in adult conversation [5]. Consequently, being cognizant of gaze has been suggested as both a proactive and reactive strategy to implement when encountering difficulties in debriefing [4]. However, many facilitators lack the knowledge, skills and awareness to effectively modulate eye contact during these challenging conversations.

Eye-tracking devices can be used as an objective tool to measure, assess and provide feedback on gaze pattern in simulation [6]. Currently, there is a paucity of research assessing gaze patterns during debriefing, with prior studies [6] focusing primarily on the learners rather than the facilitators. Attentional behaviour can be evaluated by measuring an individual’s eye movements [6]. Specifically, the visual dominance ratio (VDR) is a quotient of the percentage of time making eye contact while speaking divided by the percentage of time making eye contact while listening [7–9]. Finding the appropriate amount of eye contact is important – too much eye contact may instinctively feel rude, hostile or condescending in some situations, whereas in other contexts, longer eye contact may facilitate rapport and trust [10]. Successful interaction in a social context requires modulation of social behaviour [10]. Although modulating gaze has been described as a debriefing technique [4,11], quantitative data describing VDR during debriefing have not previously been reported.

The purpose of this pilot study was to determine the feasibility of using eye tracking during debriefing to capture facilitator gaze patterns, and to describe the VDR of facilitators in different debriefing situations. Secondarily, we aimed to explore gaze patterns among junior and senior facilitators when encountering challenges in debriefing.

Methods

This was a descriptive pilot study designed to describe the variations in facilitators’ gaze patterns in two difficult debriefing situations. A qualitative analysis of interview data obtained from the same participants has been published elsewhere [11]. Research ethics board approval was obtained from the Conjoint Health Research Ethics Board at University of Calgary (CHREB; REB 19-0100).

Outcome measures

The eye-tracking data were analysed using the Tobii Pro Lab™ software (Tobii, Sweden, version 1.114), which captures eyes movement and measures fixation time. Fixation time was defined as eyes moving at a velocity of less than 30 degrees per second. We defined the faces of the learners as area of interest (AOI), and the software measures the duration when participant facilitators have ‘eye contact’ with the actor learner’s face. We did not use actors’ eyes as AOI, as they are too small an area to be accurately captured. By manually screen the video to identify time when the participant facilitator was speaking and listening to the actor learners, we obtained all elements to calculate VDR.

Our primary outcome measure was the VDR, calculated as a quotient of the percentage of time looking while speaking and the percentage of time looking while listening [15] for each debriefing.

${\displaystyle \begin{array}{ll}{\displaystyle VDR}& =\frac{Timeoflookingwhilespeaking/Totalspeakingtime}{Timeoflookingwhilelistening/Totallisteningtime}\\ & =\frac{\mathrm{\%}oftimelookingwhilespeaking}{\mathrm{\%}oftimelookingwhilelistening}\end{array}}$

Our secondary outcome measures were: (1) percentage of time looking while speaking; and (2) percentage of time looking while listening. In comparing junior to senior facilitators, we defined the senior facilitators as having greater than 5 years of debriefing experience and having taken the advanced debriefing courses involving content like difficult debriefing, co-debriefing or peer coaching. The junior facilitators were those who did not meet these criteria.

Results

A total of 10 participants took part in the study, four of whom were senior facilitators (Supplementary material 1). A total of 30 videos with eye-tracking data were included in the analyses. Due to technical issues, two of the videos (one confrontational scenario and one emotional scenario) were incomplete, as each had less than 5 minutes of eye-tracking data recorded. We analysed the data from all videos, including the data captured from the two incomplete videos. Gaze samples is the percentage of gaze data correctly identified and measured, which ranged from 74% to 98% (mean 89.6%) in this study. An example of an eye-tracking plot for a participant interacting with learner actors is presented in Figure 1.

Figure 1:

Example of eye-tracking plot for a participant facilitator and learner actors. Each spot represents a single instance of a facilitators’ eye fixation. A fixation is defined as eyes moving at a velocity of less than 30 degrees per second. Each spot may represent a different duration of fixation.

Visual dominance ratio

Participants showed an overall low VDR across scenarios. Although senior facilitators had a slightly higher VDR than the junior facilitators, there were no statistically significant differences between the groups for all three scenarios (baseline: p = 0.17; confrontational: p = 0.76; and emotional: p = 0.61) (Table 1; Figure 2). The VDR did not change significantly between scenarios in both junior (p = 0.85) and senior facilitators (p = 0.78) (Table 1; Figure 3). The senior facilitators have a higher variability in VDR compared with the junior facilitators (Kendall’s W : junior 0.028 vs. senior 0.063, Figure 3).

Table 1:

Visual dominance ratio (VDR). For further calculation details, see text.

	Median (IQR)	Scenario			p -value*	p -value**	Friedman’s test p-value	Effect size Kendal’s W
		Baseline	Confrontational	Emotional
Debriefer	Junior (n = 6)	0.67 (0.62–0.78)	0.67 (0.55–1.13)	0.67 (0.61–0.85)	0.60	0.46	0.85	0.028
	Senior (n = 4)	0.86 (0.68–1.14)	0.77 (0.53–1.34)	0.81 (0.51–1.62)	0.72	>0.99	0.78	0.063
	Total (n = 10)	0.70 (0.65–0.98)	0.67 (0.55–1.10)	0.69 (0.61–0.96)	0.80	0.80	0.91	0.01
	p -value***	0.17	0.76	0.61

IQR: interquartile range.

* Comparison between baseline and confrontational scenarios.

** Comparison between baseline and emotional scenarios.

*** Comparison between junior and senior facilitators.

Figure 2:

Visual dominance ratio for baseline, confrontational and emotional learners.

Figure 3:

Visual dominance ratio in senior and junior facilitators.

Percentage of fixation time when listening and speaking during debriefing

The percentage of time looking at learners during listening was higher than that during speaking in all three scenarios (median percentage of time looking when listening vs. when speaking: baseline 58.77% vs. 52.63%; confrontational 68.47% vs. 50.25%; emotional 65.88% vs. 47.10%), although only the difference in the baseline scenario yielded a statistically significant result (baseline: p = 0.017; confrontational: p = 0.11; emotional: p = 0.06) (Table 2; Figure 4). The practice between junior and senior group in percentage of time looking at learners was not statistically significant (Supplementary material 2).

Table 2:

Percentage fixation time on the learners during the first 5 minutes of debriefing. For further details, see text.

	Scenario			p -value*	p -value**	Friedman’s test p-value	Kendal’s W
Median (IQR)	Baseline	Confrontational	Emotional
When listening	58.77 (50.40–81.45)	68.47 (46.90–77.71)	65.88 (40.23–76.69)	0.386	0.445	0.301	0.12
When speaking	52.63 (40.77–56.98)	50.25 (38.90–63.97)	47.10 (35.71–61.15)	0.799	0.959	0.905	0.01
p -value***	0.017	0.114	0.059

IQR: interquartile range.

* Comparison between baseline and confrontational scenarios.

** Comparison between baseline and emotional scenarios.

*** Comparison between listening and speaking phases.

Figure 4:

Fixation time for baseline, confrontational and emotional learners.

Discussion

Our study demonstrates that both junior and senior facilitators generally direct their visual gaze more towards learners when listening than speaking, resulting in an overall low VDR during debriefing. The VDRs in confrontational and emotional scenarios were slightly less than the baseline scenarios, although the difference was not statistically significant. We did not identify a significant difference in the VDRs between the junior and senior facilitators, but the senior facilitators had a slightly higher VDR than the junior. We observed an increased variability in the VDR among the senior, whereas junior facilitators had limited changes in the VDR. Below, we explore how these findings contribute to our understanding of how eye contact influences the dynamics of conversation during debriefing.

Eye-tracking technology has been applied to explore and support various aspects of healthcare provider training. A systematic review of 33 eye-tracking technology studies described how eye tracking can be used to support clinical learning, assessment and feedback in medical training [6]. Some of the studies in the review showed how the visual gaze behaviour of the learners changed along a learning curve, and eye tracking was also used to provide feedback and assessment for proficiency. However, these studies have focused on the use of eye-tracking technology in learners. To our knowledge, this is the first study to use eye-tracking devices to measure gaze fixation patterns of simulation facilitators during debriefing to explore the interaction between facilitators and learners.

The primary outcome measure, VDR, has been identified as a meaningful measure of communication and social relations [16]. Visual dominant behaviour can be influenced by personality, status, expertise, gender and team dynamics [15]. The VDR reflects the relative position on the social dominance hierarchy between the conversation partners. Those with a higher social power or more expertise typically displays a VDR close to 1 or even higher [4]. In our study, most facilitators scored a VDR less than 0.8, which may suggest their attempts to maintain psychological safety during debriefing through variation in eye contact. The senior facilitators exhibited slightly higher social dominance during debriefing, as demonstrated by higher VDRs in all three scenarios. There are many possible reasons for this observation, including but not limited to greater experience, higher standing within social hierarchy or more expertise.

The two difficult debriefing scenarios in our study (i.e. confrontational learners and emotional learners) were designed to represent breaches in psychological safety during debriefing. In such difficult debriefing situations, eye contact can be used as both a proactive and a reactive strategy. When noticing a learner is upset during debriefing, one could consider de-escalating the situation by maintaining eye contact when listening while deliberately breaking eye contact while speaking (low VDR) [4]. In our study, we found that junior facilitators failed to change their gaze patterns across the three scenarios (same median VDR in three scenarios). The senior facilitators, although maintaining a slightly higher VDR than the junior, had a lower VDR in the two difficult debriefing scenarios (median VDR: baseline 0.86 vs. confrontational 0.77 and emotional 0.81). This phenomenon suggests that some senior facilitators may be purposefully adapting fixation patterns based on learner response. This could be partly explained by the cognitive load theory. The difficult debriefing scenario imposed a high extraneous load on the junior facilitators, whose intrinsic loads were already high. The extra cognitive load made it difficult for them to have sufficient working memories to process reactive responses. Multiple other factors could potentially influence gaze patterns, such as gender [17], cultural background [18], cognitive overload [19], observation bias and facilitator preference of utilizing pen and paper during debriefing.

Variations in eye contact may or may not achieve its intended impact, as non-verbal communication is complex, dynamic and interconnected, where changes in one dimension affect other dimensions [20]. In addition, facilitators may use strategies other than eye contact to address difficulties that arise during debriefing, such as conversational techniques like validation and normalization [4,14]. This may be another reason explains why there was no significant difference noted in VDRs between junior and seniors.

Cultural background can influence debriefing practices [21–23] and gaze patterns. In North America, making good eye contact usually signifies that one is engaged and interested in the conversation, while avoiding eye contact may be interpreted as shyness or lack of self-confidence. However, in some East Asian countries, avoiding eye contact is a sign of politeness and respect, and is culturally appropriate. In some Middle Eastern cultures, there are strict rules around eye contact between the sexes, which is connected to religious laws. Different cultural backgrounds also result in different perceptions on social power and the tenets of psychological safety during debriefing. Like most of the debriefing literature, which affirms Western-centric debriefing style, our study was conducted in a Canadian centre. Extrapolating our conclusions to different cultures may not be appropriate. Future studies should examine the gaze patterns of debriefers from different cultures and how this impacts learners from cultures which differ from that of the debriefer.

Our study has several limitations. First, this research is a pilot study with a limited sample size, therefore, failing to achieve sufficient statistical power to detect a significant difference. However, the data reported in this study have built a foundation for future more strictly designed studies. Second, the project was conducted in an institute with a well-established simulation training program, which was heavily involved in simulation-based research. Most participants adhered to one framework of debriefing (i.e. PEARLS). This may limit the generalizability of the results. Third, we did not standardize the emotional change of the learner actors in each scenario. Instead, the emotion of the learner during debriefing was responsive to the facilitator. For example, if the facilitator used strategies to address the confrontational learner, the learner’s emotions de-escalated in response. We designed the study this way to reflect the circular nature of interpersonal interactions. To minimize potential biases, we decided a priori to analyse the first 5 minutes of gaze data for each participant to capture the most accurate gaze pattern coinciding with the emotional state of the learner in each scenario.

Although this study was small and did not show statistically significant differences, the use of eye-tracking technology in facilitators has opened a new pathway for simulation debriefing research. Attention to gaze patterns may be considered an advanced debriefing skill, and therefore may be more important for debriefers in the maturity phase (experts with deep understand of all concepts applied in contexts) than the debriefers in the discovery phase (i.e. novice debriefer with basic knowledge of key concepts, but little context reference) [24]. Debriefers may not be aware of their gaze patterns, and eye-tracking data could also be used as a source of feedback in debriefer faculty development. Future research could evaluate the relationship between eye-tracking data and quality of debriefing or debriefer cognitive load. Additional efforts might explore how eye-contact patterns change in response to an emotional burst of the learners (e.g. a learner suddenly cries during debriefing).

Conclusion

The use of eye-tracking devices to calculate VDR is a feasible approach to explore ways in which facilitators adapt non-verbal communication to challenges encountered in simulation debriefing. In our examination of the first 5 minutes of different difficult debriefings, we did not demonstrate a significant difference in VDR between junior and senior facilitators. More work is required to explore the association between eye gaze pattern and quality of debriefing.