Colorizing grayscale images has been a long sought-after goal in computer graph- ics and is closely related to dicult pixel prediction problems in computer vi- sion. Previous automatic approaches have failed to capture the full range and vibrancy of colors in the natural world. We have presented a method that ad- dresses this issue and is able to produce colorful and diverse results. To do so, we utilize a classication framework that models the inherent multimodality in color prediction, introduce a class-rebalancing scheme to promote the learning of rare colors, and train a bigger model on more data than past methods. We test our algorithm by evaluating a low-level per-pixel accuracy metric, along with high-level semantic interpretability and perceptual realism metrics. The results demonstrate colorization with a deep CNN and a well-chosen objective function can come close to producing results indistinguishable from real color photos.