SPEECH emotion recognition, which is defined as extracting the emotional states of a speaker from his or her speech, is attracting more attention. It is believed that speech emotion recognition can improve the performance of speech recognition systems [1] and is thus very helpful for criminal investigation, intelligent assistance [2], surveillance and detection of potentially hazardous events [3], and health care systems [4]. Speech emotion recognition is particularly useful in man-machine interaction [1], [6].