Ed is shown to be violent and abusive especially to his wife, but he would often verbally abuse the other females of the group. Carol mentions that Ed would try and show his dominance by not allowing her to wear attractive clothing and this is possibly the reasoning for Carol's short hair to make her less attractive to other males.
Ed was revealed to have paid very little attention to Sophia and when she was an infant he would often stay at a friend's house, leaving Carol to care for Sophia. While never elaborated on, it's hinted that Ed may have had some sort of sexual relationship with his daughter as Carol stated in a prayer in "What Lies Ahead" that she wished Ed to be punished for not only abusing her but also "for looking at his own daughter, whatever sickness was growing in his soul." but this is only ambiguous from Carol's own words and Lori's reaction.