Of course there would be the rare occurrence of an injured or provoked desolate beast that would charge through and destroy a small tribe.
Of course everything had both advantages and disadvantages. While keeping out a large number of desolate beasts, the shrine also caused the number of animals in its vicinity to dwindle. Hunting was not an easy task for the rabbits or deer had been hunted clean early on.