The expectation in the first term is easily approximated by View the MathML source. The second term in View the MathML source is easily computed once View the MathML source is chosen. The last term in View the MathML source corresponds to the MCMC predictive density and is evaluated as View the MathML source from the MCMC output. A total of seven different variants were proposed and evaluated in Celeux et al. (2006). Some of them are shown to provide poor results in the context of finite mixtures. One of their preferred choice is View the MathML source (together with View the MathML source not detailed here). However, they point out that “View the MathML sources can be seen as a Bayesian version of View the MathML source and […] they may under-penalize model complexity”. Consequently, a properly computed BIC is likely to select a more parsimonious model.
The choice of View the MathML source is of particular importance in the context of mixtures. Because of possible label switching, the different components of the mixture are hard to identify so choosing the posterior mean for View the MathML source can lead taking the average of several distant modes and has to be avoided. A much better choice for View the MathML source is arg maxθf(θ|y), the maximum a posteriori estimator that can be derived from the MCMC output. This remark remains valid also for the BIC criterion and we shall adopt it.
Chib (1995) has developed another method to evaluate the MCMC predictive density using a MCMC output. In Bayes’ theorem, the marginal likelihood appears as the integrating constant of the posterior density