The availability of multiple spectral measurements at each
pixel in an image provides important additional information for
recognition. Spectral information is of particular importance for
applications where spatial information is limited. Such applications
include the recognition of small objects or the recognition of small
features on partially occluded objects. We introduce a feature matrix
representation for deterministic local structure in color images. Although
feature matrices are useful for recognition, this representation depends
on the spectral properties of the scene illumination. Using a linear
model for surface spectral reflectance with the same number of
parameters as the number of color bands, we show that changes in
the spectral content of the illumination correspond to linear
transformations of the feature matrices, and that image plane rotations
correspond to circular shifts of the matrices. From these relationships,
we derive an algorithm for the recognition of local surface structure
which is invariant to these scene transformations. We demonstrate the
algorithm with a series of experiments on images of real objects.