There are several factors which prevent intruder from stealing your gem:
1. Lock
Upgrade the lock on your door to increase the number of keyholes in it. This will make it more difficult for attackers to break into your dungeon. To make an upgrade, tap on the castle icon in your dungeon (or simply swipe downwards). You're on the upgrade screen. Locate lock icon and tap "Upgrade". All upgrades need time and require gold coins.
2. Traps
Use Edit Mode to design the best defense for your dungeon, or buy the pre-made design with the best defense right out of the box. You can always change the type of traps you have by tapping "Change Traps" icon in the Edit Mode. Upgrade your traps to increase their attack damage by tapping on them. Attackers will lose more health with each death and may even leave your dungeon empty-handed eventually.
3. Gem Defense Chance
If an intruder reaches your totem, the game is not over! They need to spin a spinner and land the pointer in the green zone to steal a gem. Upgrade your Gem Defense Chance to lower the chances of the attacker of stealing a gem from your totem. This can be done in the same menu as lock upgrade, but you first need to unlock this perk.