We label data sets as D. When the distinction is important, we indicate train, validation, and test sets as:
Dtrain, Dvalid and Dtest. The validation set is used to perform model selection and hyper-parameter selection,
whereas the test set is used to evaluate the final generalization error and compare different algorithms
in an unbiased way.
The tutorials mostly deal with classification problems, where each data set D is an indexed set of pairs
(x(i); y(i)). We use superscripts to distinguish training set examples: x(i) 2 RD is thus the i-th training
example of dimensionality D. Similarly, y(i) 2 f0; :::;Lg is the i-th label assigned to input x(i). It is
straightforward to extend these examples to ones where y(i) has other types (e.g. Gaussian for regression,
or groups of multinomials for predicting multiple symbols).