Kernel tracking needs shape and appearance of the object [3]. In this tracking approach any feature of the object is used to track the object as kernel such as rectangular template or an elliptic shape. In kernel tracking object can be tracked after analyzing the motion of the kernel between two consecutive frames. Mean-shift tracking is based on the kernel tracking [20]. In this method E-Kernel tracking is used.