Every building (including destructible barrels) has a 2% chance to drop a Power Stone of some sort, and the HQ has a 25% chance. Fragments are the most common, Shards are a bit rarer and Crystals are the rarest.
Example: +100% Power Stone Chance = 4% per building and 50% for the HQ
So if you're super lucky, a base with 10 buildings could give you 10 Power Stone Crystals! And if you're unlucky, it could give you none!