There are three methodologies that have been implemented in this work. Firstly, the baseline system generally built to classify four emotions by using HMM as speech engine in emotional speech recognition. Secondly, since we defined the utterances in this work as the turn of actor’s speaking, which must have only one emotion, but sometime the actor/actress may express many emotions in the same time or they have some problem in expression about emotional ambiguity. Then some utterances may have more than one emotion within there. To solve this problem, the system was allowed to have many emotions within an utterance. In the last method, we tried to improve the accuracy by applying binary classification with HMM.