Hi,
I've also thought a lot about the differences between these words, and this is what I have come up with so far:
My impression is that:
A response is generally a reaction to something, not necessarily a verbal one; for example, you also talk of the "immune response".
The verb and noun response are followd by the preposition to.
A reply is usually a reaction or an answer to something in speech or writing, though it can also be a reaction of a different kind, like a gesture or an action.
In general, it seems to me that a "reply" is a more cognitive thing that you'll normally receive from a person, whereas a (biological, chemical, mechanical...) system can't give you a reply, but yes a response.
The verb and noun response are equally followd by the preposition to.
An answer seems to be, above all, some sort of a solution to a question or a problem.
I am not sure how the telephone fits in here, though. That's maybe just the way it is. Language is not always - if ever - logical.
The verb to answer is not normally followed by to (apart from one meaning where it means "to be called").
I would be grateful if someone could confirm or rectify this.