The most important thing of the driving is the safety of the driving. For the safe driving, the driver should be in good condition. The large percentages of the car accidents are caused by the driver's inappropriate condition to drive, i.e. drunk, injured or fell asleep, and this can be found in the statistical data from the British and American crash reports. For these reason, there are many researches focusing on the observation of the driver's physical or mental state. This research is focused on the detection of the drowsy driving. To detect the drowsy driving, a method which can assume the driver's physical condition based on the driver's head movement was developed. The driver's head position was collected by the four infra-red (IR) sensors which are mounted on the headrest of the driver's seat. From the collected head movements, the feature points and the driver's state were retrieved