Officials said Mr Abe visited the shrine in a private capacity and was not representing the government.
But shortly after, China summoned Japan's Beijing envoy to lodge a "strong protest".
Chinese Foreign Ministry spokesman Qin Gang said Beijing seriously condemned the visit.
"This poses a major political obstacle in the improvement of bilateral relations. Japan must take responsibility for all the consequences that this creates," he said.
China's angry reaction was mirrored by South Korea and Taiwan, whose Foreign Minister David Lin called for Tokyo to engage in self-reflection and avoid hurting the feelings of its neighbours.