We instead utilize a loss tailored to the colorization problem. As pointed out
by color prediction is inherently multimodal – many objects, such as a shirt,
can plausibly be colored one of several distinct values. To appropriately model
the multimodal nature of the problem, we predict a distribution of possible colors
for each pixel. Further, we explore reweighting the loss at training time to em-
phasize rare colors. This encourages our model to exploit the full diversity of the
large-scale data on which it is trained. Finally, we produce a final colorization by
taking the annealed-mean of the distribution. The end result is colorizations that
are more vibrant and perceptually realistic than those of previous approaches.