can be chosen based on the overall shape of λk(i) dependence
or from correlation analysis of loading maps, which
correspond to each of the eigenvectors, aik ≡ ak(x, y). Additionally,
Scree plot is used to correlate variance in each
component as a function of the component’s number.
Independent component analysis
Independent component analysis (ICA) is a method designed
to extract presumably independent signals mixed
within the data. Much like PCA, the output is a collection
of independent spectra and their loading maps. Unlike
PCA, however, the order of ICA components is insignificant,
and ICA takes in some input parameters and generally
takes longer to run than PCA. One of the key ICA
parameters is the number of independent components, a
decision that can be highly non-trivial to make. Another
often overlooked parameter is the number of principal
components to retain; ICA uses PCA as a filter, and for
low-dimensional data sets, or data sets with relatively few
observations, the last retained principal component plays
a huge role in the quality of the signal separation, as it
may allow or bar certain details in your data to be presented
to the algorithm.
ICA is part of a family of algorithms aimed at blind
source separation, where the objective is to ‘un-mix’ several
sources that are present in a mixed signal [42]. The
data variables are assumed to be linear mixtures of some
unknown latent variables, and the mixing system is also
unknown. ICA assumes that the latent variables will be
non-Gaussian and therefore mutually independent. The
problem of blind source separation can be modeled in
the following manner:
x ¼ As ð2Þ
where s is a two-dimensional vector containing the independent
signals, A is the mixing matrix, and x is the
observed output. As the initial step, ICA whitens the
data to remove any correlation; in other words, we are
after a linear transformation V such that if
y ¼ Vx ð3Þ
We would like to find the identity I by
Efyy0g ¼ I ð4Þ
This is possible by V = C−1/2 where C = E{xx′} giving us
Efyy0g ¼ EfVxx0V0g ¼ C−1=2CC−1=2 ¼ I ð5Þ
After the whitening, independent signals can be approximated
by the orthogonal transformation of the
whitened signal by rotating the joint density of the
mixed signals in a way to maximize the non-normality
of the marginal densities.