stage (but does employ the co-view retrieval and the reranking
model, as described in Figure 4).
In Figure 6 we report the differences in three metrics related
to watch time. The first metric is the watch time itself
(as described in Section 7.1). The second metric is the
completion rate, which measures how many of the suggested
videos were fully watched from start to finish. The third
metric is the abandonment rate, which measures the fraction
of watch videos for which no related videos were watched.
As can be seen in Figure 6, the addition of the topic retrieval
stage to the related video suggestion system results
in improvements in all three watch metrics: watch time and
completion rate increase, while the abandonment rate decreases.
As the confidence intervals shown by the error bars
in Figure 6 demonstrate, these improvements are statistically
significant.
In absolute metrics, the TransTopics method achieves
overall 1.3% increase in watch time over the baseline setup
that does not employ topic retrieval. This is an impressive
increase, given billions of hours of video watched monthly
on YouTube [2].
In addition, the TransTopics method is significantly more
effecitve compared to the IRTopics method. % change in the
watch time is 80% higher for the TransTopics compared to
the IRTopics. Similarly, % change in the completion rate is
more than double, and the % change in the abandoment rate
drops by more than 90% when comparing the TransTopics
method to the IRTopics method.
These effectiveness improvements highlight the importance
of directly learning topic transitions from user feedback. As
Figure 6 clearly demonstrates, the learned weights in the
TransTopics method can significantly improve the system
performance compared to the hand-crafted heuristic weighting
in the IRTopics method.
7.3.2 Breakdown by Video Type
In addition to the summary presented in the previous section,
it is interesting to further analyze the performance of
our methods by video type. In Table 1 we break down the
changes in the watch time metric by video category (specified
by the uploader of the video) and video age.