Fig. 1. MVCS workflow in a mobile video applicationtime point. This works well regarding the low bandwidth. As described before, the thumbnails have such a small resolution that they are loaded very fast. Furthermore, a lazy list has been implemented so that it requires even less bandwidth as only currently viewable images are loaded. Clicking on a thumbnail redirects the user directly to the corresponding scene in the video. The user can now search content much faster than in a traditional video player. This again improves the orientation for the user. If the user clicks on an image he is directly redirected to the corresponding time point in the video. Furthermore, the seek bar focuses the current scene and scrolls automatically.
Fig. 2. Video stream browsing based on video segmentation and automatically generated metadata
Second, the tag list (right) consists of tags which have been added manually by the user himself, by other users, or generated automatically. Like the thumbnails the tags are ordered by timely occurrence in the video. If a user clicks on a tag, the video stream goes directly to the corresponding position. Both components, i.e. the segment-based seek bar and the tag list are implemented to overcome the mobile UX problem of video browsing on a mobile device. Finally, the third part of mobile UX improvement contains the video player itself. As device information including screen size is sent to the cloud the necessary zooming ratio can be calculated. Depending on the screen size of the device
a zooming ratio is calculated and depending of the objects a center position of the zooming field is defined. The red rectangle symbolizes the zoomed video. Two persons are recognized by the object recognition service and therefore the zooming field is set at this position. The user just sees this part of the video and can better concentrate on the persons. For future implementations more enhanced classifiers or feature descriptors can be used (e.g. football, etc.) to enable personalized zooming.
V. MOBILE VIDEO CLOUD SERVICES
The mobile services architecture is based on the previous workflow. As shown in Figure 3, the mobile client and cloud components are communicating with each other using different protocols for different purposes. RTP is used for video up and down streaming. The extensible messaging and presence protocol (XMPP) is used for the exchange of metadata, segments information, and device information. HTTP is required for file transfers like video upload to the cloud. Realizing the different functionalities of the MVCS client various handler are required. The MP4 handler is an important part of the video streaming functionality. As described before the RTP protocol is used for video streaming. To ensure good compatibility with various streaming servers the outgoing video stream should consist of a video encoded by the H.264 video codec and an MP4 video container. The metadata handler is a very simple handler. It manages the fetching of the video segments and tags via the XMPP connector and parses the XML segments. Furthermore, it sends new tags via the XMPP connector. Additionally, it manages sending the video metadata information like title and description. The lazylist handler is an important part of mobile user experience as it reduces data transfer and memory usage.
Fig. 1. MVCS workflow in a mobile video application
time point. This works well regarding the low bandwidth. As described before, the thumbnails have such a small resolution that they are loaded very fast. Furthermore, a lazy list has been implemented so that it requires even less bandwidth as only currently viewable images are loaded. Clicking on a thumbnail redirects the user directly to the corresponding scene in the video. The user can now search content much faster than in a traditional video player. This again improves the orientation for the user. If the user clicks on an image he is directly redirected to the corresponding time point in the video. Furthermore, the seek bar focuses the current scene and scrolls automatically.
Fig. 2. Video stream browsing based on video segmentation and automatically generated metadata
Second, the tag list (right) consists of tags which have been added manually by the user himself, by other users, or generated automatically. Like the thumbnails the tags are ordered by timely occurrence in the video. If a user clicks on a tag, the video stream goes directly to the corresponding position. Both components, i.e. the segment-based seek bar and the tag list are implemented to overcome the mobile UX problem of video browsing on a mobile device. Finally, the third part of mobile UX improvement contains the video player itself. As device information including screen size is sent to the cloud the necessary zooming ratio can be calculated. Depending on the screen size of the device
a zooming ratio is calculated and depending of the objects a center position of the zooming field is defined. The red rectangle symbolizes the zoomed video. Two persons are recognized by the object recognition service and therefore the zooming field is set at this position. The user just sees this part of the video and can better concentrate on the persons. For future implementations more enhanced classifiers or feature descriptors can be used (e.g. football, etc.) to enable personalized zooming.