This paper proposes an interactive e-learning system using pattern recognition and augmented reality.The goal of proposed system is to provide students with realistic audio-visual contents when they are leaning.The proposed e-learning system consists of image recognition,color and polka-dot pattern recognition,and augmented reality engine with audio-visual contents.When the web camera on a PC captures the current page of textbook, the e-learning system first identifies the images on the page,and augments some audio-visual contents on the monitor. For interactive learning,the proposed e-learning system exploits the color-band or polka-dot markers which are stuck to the end of a finger. The color-band and polka-dot marker act like the mouse cursor to indicate the position in the textbook image.Appropriate interactive audio-visual contents are augmented as the marker is located on the predefined image objects in the textbook.The proposed e-learning system was applied to the educational courses in the elementary school,and we obtained satisfactory results for real applications.We expect that the proposed e-learning system is popular when the educational contents and scenarios are sufficiently provided.