When the drilling and fracking process is done well and no problem occurs, the fracking fluid should never come in contact with any ground or surface waters. However, on a non-trivial proportion of wells problems do occur. One significant issue is the failure of the steel or concrete well casings, allowing fracking water to enter the ground much closer to the surface, leading to ground and surface water contamination. For example, the U.S. Environmental Protection Agency determined that well water contamination for residents of Pavillion, Wyoming, was due to the movement of gas and fracking fluids away from wells and into the groundwater system.
Worse than the fracking chemicals, however, might be the flowback or produced water recuperated after fracking, as it contains very high concentrations of salt, heavy metals, and sometimes radioactive material. These elements originate from the rock formation itself, are much more concentrated in the waste water than the chemical additives are, and probably should be much more of an environmental concern.