Data Collection and Modeling
To understand the relationship between gaze and personality
in a motivational task, we sought to answer the following questions.
How do people use their gaze when attempting to motivate
others and increase compliance? What is the relationship
between their use of gaze and their respective personalities?
To answer these questions, we conducted a human-human
data collection study with four participant dyads, obtaining
more precise measurements of gaze behavior than what is
traditionally presented in the social-sciences literature. We
chose the Tower of Hanoi puzzle for participants to complete
collaboratively. The goal of this puzzle is to move a number
of colored blocks from one location to another while following
some simple rules. This task was chosen for its mix of
cognitive (solving the puzzle) and physical (actually moving
the pieces around) elements, mapping well to tasks commonly
used in physical and cognitive rehabilitation. The task can
also be broken down into two repeating phases common to
rehabilitation activities: (1) the actual execution of the task,
which we will refer to here as the in-task phase, and (2) the
time between tasks when the therapist must provide encouragement
to persist with the task, which we will refer to here
as the between-task phase.
Participants filled out the Big Five inventory prior to participation
to determine their position on the extroversionintroversion
spectrum [19]. The Big Five questionnaire contains 44 items on a five-point rating scale that ask the participant
to rate their agreement or disagreement with statements
about their own personality and activities. Eight of these items
contribute to the extroversion dimension of the participant’s
overall personality score. These items have good internal reliability
(Cronbach’s = .88). Participants scoring lower than
2.5 on the extroversion dimension were labeled as introverted,
and those above 2.5 were labeled as extroverted.
In each dyad, one participant was assigned to be the instructor
and the other the worker. Each of the four dyads covered
one of the four possible combinations of participant personality
and role. The experimenter first explained the puzzle
to the instructor without the worker present. Next, the instructor
practiced solving the puzzle with the experimenter.
The instructor was required to successfully solve the puzzle
a number of times in front of the experimenter to prove that
they were comfortable with the task. Then, the instructor was
asked to carry out the following procedure: (1) explain the
task to the worker, (2) monitor the worker as they complete the
task, (3) provide encouraging feedback during the puzzle solving,
(4) motivate the worker to keep working between puzzle
tasks, and (5) correct workers when they make a mistake. The
worker was told by the experimenter that everything would
be explained by the instructor, but that they were welcome to
work on the task for as long as they wished and that it was up
to them to decide when they would like to stop.
The two participants sat at a table facing each other, with
the puzzle between them (Figure 2). Over-the-shoulder view
cameras recorded the gaze of each participant, and a side
camera with wide-angle view was used for recording the entire
task. The compliance of the worker to the instructor was
measured in three ways: (1) total time spent solving puzzles,
(2) total number of puzzles completed, and (3) total number of puzzle pieces moved. Worker compliance in each dyad, as
well as the percentage of each participant’s attempt to engage
in mutual gaze with their partner, is presented in Table .
All videos were coded for participant gaze behavior. Participant
gazes were recorded and labeled for two targets: the other
participant and the shared workspace. The mean and standard
deviation of these gaze lengths are presented in Table .
We observe three trends from the data. First, extroverts seem
to be attempting to engage in more mutual gaze with their partner
than do introverts. This relationship between extroversion
and gaze behavior has been similarly demonstrated in previous
research, including a study of dyadic interviews in which it
was found that extroverts gaze at their interviewer more than
introverts do [18]. Second, there is more mutual gaze between
puzzle phases when the instructor is attempting to motivate
the worker to solve more puzzles and much less mutual gaze
during the actual puzzle completion. Third, there is some preliminary
indication that personality matching is an effective
strategy for increasing compliance, as personality-matching
dyads exhibited longer time-on-task than the mismatching
personality dyads. This point is explored further in the experimental
evaluation presented later in this paper.
The results presented in Table are used to generate two models
of gaze behavior for robots, one to express an extroverted
personality and the other to express an introverted personality.
The presented means and standard deviations are used
to create normal distributions of gaze lengths that the robot
draws from when planning and executing gazes toward the
user and toward the task space. In an expression of the extroverted
model, the robot gazes into the face of the user more,
while the introverted model generates more gaze toward the
task space. In both models, more gaze is generated toward
the user in the motivational between-task phase than in the
in-task phase, which involves monitoring the user’s actions.
For example, an extroverted robot drawing from the distributions
in Table might generate a four-second gaze toward the
user in a between-task phase, followed by a one-second gaze
toward the task space. This sequence of long user fixations
and short task fixations—randomly generated according to the
distributions—would repeat until the start of the next in-task
phase. At this point, the extroverted robot might generate a
four-second gaze toward the task space followed by a two-anda-
half-second gaze to the user. This cycle of gaze shifts would
repeat throughout the in-task phase.