The original appearance was altered in the Middle Ages when a fortress was constructed in the building by the Pierleoni family, and yet again in the 16th century, when it was transformed by Baldassare Peruzzi into the residence of the Savelli family.