As you would when calculating the sample variance of any random variable, you must divide by the degrees of freedom. The degrees of freedom represent the number of independent values used when producing the variance estimate; in other words, it’s the number of observations minus the number of estimates that must be produced before the sample variance is calculated. In the case of a residual variance, you first need to estimate the regression function before you can calculate each residual. That means you lose degrees of freedom p plus the intercept (or constant).