Fig. 3. Google Audio Indexing page on labs.google.com/gaudi. Snippet appears on mouse over of a marker in the playbar.
and a rough transcription suffices. For a more detailed overview, we return the ten best snippets, selected primarily on the number of token hits in the snippet and the confidence scores associated with those hits.
To facilitate rapid video navigation, the snippets should index into the video and provide points to start playback at their occur- rences. Our UI implements this through custom javascript controls of the YouTube player letting any element on the page seek to arbitrary points in the video. As shown in figure 3, due to space limitations in the gadget we have integrated the snippets with the player controls, placing markers on the playbar at the start time of each snippet. Clicking on the marker initiates playback two seconds be- fore the start time of the first snippet word. The added offset pre- vents missing the snippet audio due to timing imprecision and/or seeking irregularities. The yellow markers on the playbar provide a good one-glance overview of the frequency of the query terms in the video. Because the markers correspond to relevant video sections, they provide a high level overview token hits.
The playbar is the logical point of integration of text with video timing, but not ideal due to the combination of a small playbar size and a large density of transcript information. Especially for longer videos, a pixel in the playbar width may correspond to 10 or more seconds, causing some markers to overlap which detracts from the mouseover and browsing experience. We have experimented with merging the overlapping markers, but the solution was clumsy since the underlying video segments are actually temporally distinct. Therefore, we show only one of the overlapping snippets, and recalculate overlaps whenever the player is resized. On our labs page, we have utilized the extra space to give immediate access to all the snippets as shown at the bottom of figure 3.