2. Moisturize. Many acne products contain ingredients that dry the skin, so always use a moisturizer that minimizes dryness and skin peeling. Look for "noncomedogenic" on the label, which means it should not cause acne. There are moisturizers made for oily, dry, or combination skin.