Abstract-Requested by intelligentization of video surveillance system, the paper proposes a moving objects detection program based on Embedded video surveillance. With the help of the transaction of Servfox streaming media on ARM-Linux operation platform, the video collection and transmission of USB camera is achieved. The paper adopts the improved algorithm of the updated background model of inter-frame difference to obtain the moving objects in the client. The method reduces influence of moving objects to the background model. The experiment shows that the background image can be obtained accurately, and the moving objects recognition is achieved effectively.