Connect an ion exchange water softener to your water supply. These devices contain a material, such as sodium, that combines with the iron chemically, thus removing the rust from the water. Water softeners also use ion exchange to remove a variety of mineral compounds from “hard” water. Some manufacturers recommend that you only use water softeners for drinking water with iron concentrations below two parts per million (ppm.