But I've had the same experience as you, while trying to learn Spanish. I went to movies, watched TV, listened to the radio. 90% of the people where I work speak Spanish and I have no idea what they're saying, unless it involves phrases which I've learned by listening to language records and following the "script" and translation on paper. And Spanish pronounciation is as easy as it gets! I can imagine how you guys suffer with English! I can read Spanish, and I can make myself understood when speaking to a native speaker of Spanish. (My French falls into the same category.)
Spanish is my wife's native lanaguage, but she refuses to speak it in front of me.
Anyway, I share your experience re diction. I can listen to some radio anouncers and repeat back a phrase word for word, but such speakers are few and far between. And it's not a matter of speed. It's a matter of clarity, voice quality, and perhaps "crispness." I have more trouble hearing the female voice, but I can't explain why. The delivery seems more blurred. And a bass voice is easier for me to understand than a higher male voice.