The "yes" campaign had the backing not just of President Santos but of a wide array of politicians both in Colombia and abroad, including UN Secretary General Ban Ki-moon.
But there was also a vocal campaign for a "no" vote, led by former Colombian President Alvaro Uribe.
Polls conducted ahead of Sunday's vote suggested a comfortable win for the "yes" campaign.
But in a surprise result, 50.2% of voters rejected the agreement compared with 49.8% who voted for it.
The difference with 98.98% of the votes counted was less than 54,000 votes out of almost 13 million ballots.
Turnout was low with fewer than 38% of voters casting their votes.