3.4.5 Save Percentage
Since we were interested in looking at the relationship between
anxiety and performance, we kept track of the save percentage
throughout the study. The save percentage was computed using the
participant’s actual performance in the system (not by the displayed
saves in the high anxiety conditions). In order to be a SAVE, the
participant needed to move in the correct direction and within the
allotted time threshold. The direction of movement was determined
by looking at the change in x-value from the head tracker (see 3.6 for
details on the hardware). It should be noted that participants were
instructed to move their whole body during the experiment – not just
their heads (see 3.6). We wanted the task to be difficult but possible,
and since individuals vary greatly in their reaction time, we needed
to calibrate the system to each participant. We leveraged the two
training phases (see 3.8 for details on the procedure) to calculate the
threshold. In each trial, we computed the time it took the participant
to move 0.15m in the correct direction (this value was chosen after
experimentation with a variety of users). At the end of each training
phase we calculated the average of all successful trials. The lower of
the two was used during the experimental conditions.
3.4.6 Reaction Time
In addition to save percentage, we also kept track of user reaction
time. While save percentage is a more holistic variable in terms of
performance (since it incorporates both direction and timing), it does
not offer any differentiation among trials that are saves, or among
trials that are misses. We therefore also kept track of reaction time,
since it offers a finer-grained representation of the performance
between trials. In our experiment, reaction time refers to the amount
of time that passed from the onset of the kick to when the participant
had moved 0.15m in either direction. Reaction time can be an
indicator of anxiety itself [8], hence why we calculated the value for
all trials and not just SAVES.
3.4.7 Background Questionnaire
We included a background questionnaire as part of the preexperiment
questionnaire to gain information about the participants’
demographics and competitive backgrounds.
3.4.8 Post-experiment Interview
We included an informal post-experiment interview to get detailed
information about the reactions and emotions participants
experienced throughout the study. The interview asked participants
to describe the emotions they experienced, to discuss their
experience with the system.
3.5 Task
The task for the application was to defend against penalty kicks in a
simulated soccer shootout. In each condition the participants faced
15 kicks, between which the application would fade to black and
then fade back in. In real-life shootout situations the goalkeepers
from both teams alternate defending against kicks. We choose to
fade out the scene between trials to better accommodate a single
user. The participants stood in the middle of the VisCube display in a
ready position with knees slightly bent, and waited until the ball was
kicked before making any move. Once the participants could see the
direction of the kick, they were instructed to take a quick step in the
correct direction, leading with their shoulder and head so the head
tracker would capture the movement. Participants were instructed to
return to the center of the display after each trial.
3.6 Apparatus
The study was conducted using the Virginia Tech Visionarium
VisCube. The facility is a four-screen CAVE-like projection system,
with each wall having dimensions of 10’ x 10’, and featuring a
resolution of 1920 x 1920 pixels. The application was projected in
stereo with head tracking using an Intersense IS-900 tracking system.
Users wore Infitec stereo glasses, with a wireless head-tracker
mounted to the top frame. The x-value of the head tracker was used
to determine the participants’ movements throughout the study. An
Apple iPad was used to capture questionnaire responses.
The application used the Instant Reality framework [39] with the
code was written in X3D [40]. All the 3D objects and characters for
the application were modeled using Google SketchUp, imported into
3D Studio Max, and animated using the biped tool. The primary kick
animation was acquired from the Carnegie Mellon University
Motion Capture Database [41] and edited using Bvhacker freeware
[42] to correct misaligned posture. All other character animations for
the application were created manually. The two soccer ball
animations (one left and one right) were created using the 3D Studio
Max Havoc 3 physics engine to achieve realistic flights. All the
elements were exported into VRML format, and then converted to
X3D format using the online X3D encoding converter.
3.7 Participants
After receiving approval from Virginia Tech’s Institutional Review
Board, participants were recruited by placing an advertisement in the
weekly email newsletter sent to all university graduate students. In
total 30 participants were recruited, and 28 of the participants
completed the study. Participants were paid in order to encourage
them to focus and stay motivated throughout the entire study. In a
deployed VR training system, this motivation would be internal
(desire to improve), but since our participants were not actually
training for goalkeeper performance, we felt external motivation was
needed. All participants were paid according to the payment
schedule ($12 base + $0.15/save, for a total possible payout of $30).
Of the 28 participants that completed the study, 25 were included
in the analysis (the first three were pilot participants). All of them
were graduate student at our university, and ages ranged from 22 to
32. Nine of the participants were female, and 15 were male. Twentythree
participants had competitive sport experience, 15 had
competitive soccer experience, and seven had goalkeeper experience.
3.8 Procedure
Upon arrival each participant was greated and asked to fill out the
IRB consent form. The experimenter helped the participant put on
the HR and GSR monitoring equipment, and then asked the
participant to sit and fill out the pre-experiment questionnaire.
The participant was then introduced to the VisCube display and
the system controls, and was asked to put on the stereo glasses with
the mounted head tracker. The participant then completed a twostage
training process. Both stages used the low anxiety, high FOR,
low SF environment (chosen to allow participants to become familiar
with the full FOR of the display and to allow full focus on the
training task without added distractors and anxiety-inducing
elements). The first training stage began with detailed instructions on
how to interact with the application. The participant was told to stand
in the middle of the display and wait until they saw the kick, and
then to take a quick step in that direction. In real-life situations
goalkeepers often react before the kick, and use cues from the kicker
to guess the correct direction. However in our application we were
interested in looking at reaction time, so we chose to require
participants to wait until after the kick. The avatar animation was the
same for both the left and right kick, so there were no cues to
distinguish the direction. In this phase of training, the system
provided feedback messages of either “Good,” “Too early,” “Too
late,” or “Wrong direction.” This training phase continued until the
participant achieved 15 “Good” trials.
During the second training stage, the application functioned in a
manner similar to the main experimental conditions. No detailed
feedback was given to the participant, only a message of either
SAVE or MISS after each trial. To be a SAVE, the participant had to