Every tracking system demands face detection either in every frame or when the face comes into visible in the video [7]. A common method for face tracking is to use information in a single frame. Meanwhile some face detection methods make use of the time-based or sequential data[5][6] generated from a series of frames to minimize the number of false detections. This sequence data is normally in the form of frame variance, which draw attention the changing regions in successive frames. Given the face regions in the image, it is then the tracker’s duty to do object correspondence from one frame to the next to generate the detection