Scott was born in London, the only child of Antarctic explorer Robert Falcon Scott and sculptor Kathleen Bruce. He was only two years old when his father died. Robert Scott, in a last letter to his wife, advised her to "make the boy interested in natural history if you can; it is better than games."[3][4] He was named after Sir Clements Markham, mentor of Scott's polar expeditions, and his godfather was J. M. Barrie, creator of Peter Pan.
He was educated at Oundle School and Trinity College, Cambridge, initially reading Natural Sciences but graduating in the History of Art in 1931. Whilst at Cambridge he shared digs with John Berry and the two shared many views.[5]