Buddhism declined when the dynasties patronizing it declined,
following White Hun and the early ArabIslamic
invasions
such as that of Muhammad bin Qasim.
A strong revival occurred under the Pala Empire in the northeast of the subcontinent (where the temple is situated).