As an alternative to the preceding supervised approaches to social state recognition and social skills execution, which require labelled data, we have also developed a non-parametric Bayesian framework for automatically inferring social states in an unsupervised manner [13], which can be viewed as a natural fusion of multimodal observations. This approach makes use of the infinite POMDP method [2], which does not require advance knowledge of the size of the state space, but rather lets the model grow to accommodate the data.