17.4.3 Choosing Weights Using Poisson Regression
There are three reasons why we cannot use the coefficients bkℓ resulting from the
Poisson regression model as weights directly. The first reason is that dissimilarity
scores and attributes do not have the same scale. Second, uncertainty about the
correctness of the coefficient is not taken into account when using bkℓ directly. Although
the value of a coefficient can be relatively high, it can still be unimportant.
Consider, for example, a dummy attribute having very few 1’s and a high coefficient
value. Then, this high impact of the coefficient is only applicable to a limited number
of items and its total importance is limited. By taking the uncertainty we have
into account, we can correct for this. Third, weights should always be equal to or
larger than 0, while bkℓ can also be negative.
The first two problems can be overcome by using the t-value