Face recognition is still a challenging problem because of large intra-class variability, small inter-class variability, and the presence of lighting variation. To deal with these difficulties, an illumination compensation method, adaptive singular value decomposition in the two-dimensional discrete Fourier domain (ASVDF) and an efficient brightness detector for lighting detection, for face image enhancement are proposed in this paper. The proposed enhancement algorithm involves three steps: In the first step, uniform lighting is rapidly distinguished from lateral lighting in the image by using the brightness detector, which is based on the percentage ratio of pixels among the three RGB color channels. ASVDF is then globally performed for the uniform lighting image, whereas ASVDF is applied block-by-block for the lateral lighting image. In addition, to reduce computing time, a region-based ASVDF method is introduced; the ASVDF method is applied to four regions of the face image. Experimental results for the CMU-PIE, Color FERET, and FEI face databases show that the method considerably improves the quality of face images, even lateral lighting, thereby improving the accuracy of face recognition substantially.