Pepper is widely planted and used all over the world as fresh vegetable and spice. Genetic and morphological
information of pepper are stored through seeds. Determination of seed variety is crucial for correctly identifying genetic
materials. Pepper varieties cannot be easily classified even by an expert eye due to the very small size of seeds and visual
similarities. Hence, more advanced technologies are required to determine the variety of a pepper seed. A classification
method was proposed to discriminate pepper seed based on neural networks and computer vision. Image acquisition was
conducted using an office scanner at a resolution of 1200 dpi. Image features representing color, shape, and texture were
extracted and used to classify pepper seeds. By calculating features from different color components, a feature database was
constructed. Effective features were selected using sequential feature selection with different criterion functions. As a result
of the feature selection procedure, the number of the features was significantly reduced from 257 to 10. Cross validation rules
were applied to obtain a reliable classification model by preventing overfitting. Different numbers of neurons in the hidden
layer and various training algorithms were investigated to determine the best multilayer perceptron model. The best
classification performance was obtained using 30 neurons in the hidden layer of the network. With this network, an accuracy
rate of 84.94% was achieved using the sequential feature selection and the training algorithm of resilient back propagation in
classifying eight pepper seed varieties.