Comparison of Estimation Methods
Generally, the parameters estimated using weighted least squares are not the same as those estimated using maximum likelihood except for the special case in which the data are normally distributed (as assumed in a standard regression analysis). However, for large sample sizes both methods are very similar and converge to parameters that are asymptotically consistent and have minimum variance (assuming the model is true).
Simulation can be used to compare the recovery of true parameters from sample data for maximum likelihood and Bayesian methods. In this particular example, a large scale simulation was used to evaluate parameter recovery of individual parameters for maximum likelihood and Bayesian methods when applying the PVL model to IGT choice data (Ahn et al., 2009).
In Figure 4.3, each column of histograms represents one of the PVL model parameters. The first row of histograms shows the true distribution of parameters (embedded in the simulation); the second row shows the distribution of Hierarchical Bayesian estimates for the individual level parameters; the third row shows the distribution of the hierarchical Bayesian estimates for the group level or hyper parameters; the four row shows distribution of Bayesian estimates when applied separately to each person (not hierarchical); the fifth row shows the distribution of the maximum likelihood estimates for each individual (the red line in the last row shows the maximum likelihood estimate when fitting all simulated participants using the same parameters). The main comparisons to make are among the top, second, and the last rows. As shown in Figure 4.1, the hierarchical Bayesian method recovered individual parameters in the simulation much better than maximum likelihood method in this dataset.