'Keep at bay' (sometimes used as 'hold at bay') is one of those expressions that we are likely to know the meaning of because we have picked it up from colloquial use in our youth and worked out the meaning from the context it was used in. The nature of how we learn language allows us to gain a knowledge of what an idiom means without necessarily knowing the meaning of the words contained in it - which I guess is why pages like this one get readers.