Recall that p-type material has trivalent impurities and n-type material has pentavalent impurities.
The trivalent impurities exert lower forces on the outer-shell electrons than the pentavalent impurities.
The lower forces in p-type materials mean that the electron orbits are slightly larger and hence have greater energy than the electron orbits in the n-type materials.