OK, so... Well. Let's start from the alphabet. We have 33 letters divided into 10 vowels, 21 consonants and 2 letters which do not designate any sounds. It's ъ (твёрдый знак/hard sign) and ь (мягкий знак/soft sign). Although they are not sounds of their own, they influence the way you read the neighbouring letters. This is not easy to explain, because these signs note the difference that doesn't exist in English.
So the soft sign basically means that the previous consonant must be pronounced in a soft way. For example, мел and мель. Some letters, like я, ю, ё also implicate the softening of the consonant. When this is not desired, it must be "neutralised" by the hard sign. Sometimes hard sign is used in composite words to mark the border and hint correct emphasis (ex.: трёхъярусный). Hard signs can also be encountered in transcripts of some foreign names. In this case spelling is just learned. Plus there are exceptions, where you just spell it one way or the other, though the pronunciation difference has been lost.