Customization Bias
The study design is a 2x2 design with a between-subjects
factor (customization) and a within-subjects factor (recommendation
quality). To test for the effect of customization on
agreement with the DSS, I fit multilevel regression models
with the experimental factors as fixed effects and a random
effect for each subject to account for the repeated measurements
in the design. For assessing the binary measure of winner
agreement, I used multilevel logistic regression. These
models are described in Table 1. The intercept of these models
can be interpreted as the estimated degree of agreement
when subjects do not customize the DSS and receive a good
recommendation. The coefficients represent differences from
this baseline group, and the table shows the standard error
below each estimate in parentheses. Figure 2 shows the models’
estimated degree of agreement for all four combinations
of the factors. This figure converts the log odds estimated by
model 1 into the probability of a subject agreeing with the
DSS.
Model 1 shows a statistically significant effect of customization
on the binary measurement of agreement. Customization
users were overall more likely to predict the same team to
win as the DSS. Subjects were less discerning of poor recommendations
when they had customized the DSS, as they were
more likely to agree with the DSS when receiving a poor recommendation
than those who did not customize. Conversely,
customization users were better at discerning good recommendations
as well, being more likely to follow good recommendations
than the control group.
Model 4 tests these effects in terms of score agreement.
Again, customization users were biased towards agreeing
with the recommendation. On average, customization users
predicted scores that were 0.9 runs closer to the DSS recommendation
than non-customization users. When the recommended
scores were accurate, subjects predicted 0.96 runs
closer on average than when the recommendation was inaccurate.
Models 2 and 3 in Table 1 add the difficulty of the game to the
models to see whether the difficulty of the game would influence
how subjects interpreted recommendations and whether
this would be different between the two conditions. Since
subjects had a higher probability of receiving a poor recommendation
for difficulty levels 1 and 4 than for levels 2 and
Customization Bias
The study design is a 2x2 design with a between-subjects
factor (customization) and a within-subjects factor (recommendation
quality). To test for the effect of customization on
agreement with the DSS, I fit multilevel regression models
with the experimental factors as fixed effects and a random
effect for each subject to account for the repeated measurements
in the design. For assessing the binary measure of winner
agreement, I used multilevel logistic regression. These
models are described in Table 1. The intercept of these models
can be interpreted as the estimated degree of agreement
when subjects do not customize the DSS and receive a good
recommendation. The coefficients represent differences from
this baseline group, and the table shows the standard error
below each estimate in parentheses. Figure 2 shows the models’
estimated degree of agreement for all four combinations
of the factors. This figure converts the log odds estimated by
model 1 into the probability of a subject agreeing with the
DSS.
Model 1 shows a statistically significant effect of customization
on the binary measurement of agreement. Customization
users were overall more likely to predict the same team to
win as the DSS. Subjects were less discerning of poor recommendations
when they had customized the DSS, as they were
more likely to agree with the DSS when receiving a poor recommendation
than those who did not customize. Conversely,
customization users were better at discerning good recommendations
as well, being more likely to follow good recommendations
than the control group.
Model 4 tests these effects in terms of score agreement.
Again, customization users were biased towards agreeing
with the recommendation. On average, customization users
predicted scores that were 0.9 runs closer to the DSS recommendation
than non-customization users. When the recommended
scores were accurate, subjects predicted 0.96 runs
closer on average than when the recommendation was inaccurate.
Models 2 and 3 in Table 1 add the difficulty of the game to the
models to see whether the difficulty of the game would influence
how subjects interpreted recommendations and whether
this would be different between the two conditions. Since
subjects had a higher probability of receiving a poor recommendation
for difficulty levels 1 and 4 than for levels 2 and
การแปล กรุณารอสักครู่..
