Traffic sign detection has been tackled by various approaches such as rule-based method, and hybrid algorithms. Most works extracted region of interest (ROI) with color information and were sensitive to weather or illumination effect to verify the traffic sign [1], [2]. It is challenging to exactly extract ROI regardless of such environmental conditions. For example, the segmentation of traffic signs using color pixel is not working for complex or overlapped scenes and requires additional pre-processing procedure. Also, the previous works tested their approaches on some simple scenes or highways [3], [4]. Recently, there are several detection works using machine learning algorithms instead of the color-based ROI approaches. For example, convolutional neural network [5] and support vector machines have been used for detection over real- world scenes .