Click on the new channel’s miniature icon while holding the Control/Command key to select the content.
Activate the “RGB channel” (top-most), and switch back to the Layers palette. When the duplicated background is selected, click on the “Add Layer Mask” icon.
Click on the Layer Miniature icon, and select Filter → Blur → Surface Blur from the menu. Play around with the Radius and Threshold sliders until the noise has been reduced as much as possible. Thanks to the mask you created, the contours are safe.