X has, for centuries, been a recognised abbreviation for Christ. It represents the first letter of the Greek word khristos - I can't do the Greek letters, sorry.
As it would be unimaginable to write Christmas as Christ'mas (although it shouldn't surprise me to find plenty of examples on the Greengrocers stall:) ), so it should be unimaginable to write Xmas as X'mas.