Comfortable travel is an essential issue of Intelligent Transport Systems (ITS). However, the driver's behavior and the road condition affect the comfort of the passenger's riding experience while they are traveling. In this paper, we propose a system named Riding Experience Sensor (RESen) for sensing and evaluating the riding experience, based on crowd sourcing by smart phones. We utilize the acceleration sensor and gravity sensor for sensing with arbitrary orientations of smart phones. We partition the riding experience into horizontal and vertical for evaluation. Thus, based on the driver's historical trajectories, the system can provide feedbacks for improving driving by finding the anomalies along these trajectories. Based on the map, which has evaluated the comfort of each road, the system can provide a comfortable travel plan for query users.