Rhodes was born in 1853 in Bishop's Stortford, Hertfordshire, England. He was the fifth son of the Reverend Francis William Rhodes and his wife Louisa Peacock Rhodes. His father was a Church of England clergyman who was proud of never having preached a sermon longer than 10 minutes. His siblings included Francis William Rhodes, who became an army officer.