In this work, we address the problem of gender recognition from face images taking into account the memory and the running time issues and by using a relatively small training dataset. In particular, we design a CNN-based ensemble model obtaining the state-of-the-art performance on gender recognition from face images in the most stringent conditions. We use a publicly available dataset of face images to train our CNN-model obtaining the highest recognition accuracy with about 10 times less training data than the state-of-the-art authors [11]. Our model is also minimized both in terms of the running time and the memory requirements making its usage possible even on devices with a http://dx.doi.org/10.1016/j.patrec.2015.11.011 0167-8655/© 2015 Elsevier B.V. All rights reserved. 60 G. Antipov et al. / Pattern Recognition Letters 70 (2016) 59–65 limited memory and without dedicated graphical processors for computations.