Gamma rays can interact with cells in the body and make them non functional or they can mess up a cell's DNA and cause mutations when it divides. There are many, many cells in the body so you can lose some of them without any harm. If you lose too many you can get sick or with massive doses, can die. There is always a small amount of background radiation coming from cosmic rays from outer space and from radioactive nucleii found in the earth's environment. Directed gamma radiation is used in many cancer treatments.
Here is US EPA web site that has lots of useful information on gamma ray