When important decisions have to be made, society often places its trust in groups
of people. We have parliaments, juries, committees, and boards of directors, whom we are
happy to have make decisions for us. It is generally accepted that “two heads are better than
one”, and that bad choices can be avoided by encouraging people with different viewpoints
to reach mutually agreeable decisions. The Condorcet Jury Theorem [156] proposed by
the Marquis of Condorcet in 1784, studied the conditions under which a democracy as a
whole is more effective than any of its constituent members. In 1818, Laplace [30] observed
that a suitable combination of two probabilistic methods will perform better than either of
the components. From these first insights, a number of issues have arisen which have been
studied extensively by a number of different research groups, among which is the Machine
Learning community, the target audience for this thesis.