Subjects wagered an average of 4.22 confidence points per
round (S.D. 2.5). To see whether customization influences
DSS users’ confidence in their decisions, I fit a similar multilevel
regression model as model 4 above with customization
and recommendation quality as independent variables and the
number of confidence points wagered by the subject as the
dependent variable. This model showed no statistically significant
effect of customization on confidence in decisions.
I also tested the effect of customization on subjects’ overall
decision quality to see whether customization bias led to
overall differences in decision making quality. I defined decision
quality as the number of points earned from the prediction
of a game, including points earned from confidence
wagers, because the incentive of the game was to score as
many points as possible. The scoring structure created a bimodal
distribution because of the large number of points lost
when choosing the wrong winner and the 3 to 1 return on
confidence points. To correct this, I rank transformed each
prediction’s points earned compared to all other rounds from
the experiment, with a rank of 1 being the lowest number of
points. Table 2 shows the average rank in each of the four prediction
categories. Subjects made the best overall decisions
when they customized a good recommendation, and the worst
decisions when they customized a poor recommendation, and
all terms including the interaction from the model were statistically
significant (p < .01). I supplemented this analysis
by simply comparing the point totals from all twelve rounds
between subjects, measuring the total performance of subjects
in the customization condition agains the control group.
The mean number of points earned per subject was 360.2
(S.D. 46.4). An OLS regression indicated that customization
subjects earned 19.3 more points than those who didn’t customize
over the whole experiment.