He added that he had made the decision after being assured that it was still possible to complete construction of a new design in time for the Olympics.
Under the original plans, Tokyo's stadium would have been bigger and more expensive than any of its recent predecessors.
It drew increasing criticism as estimated costs spiralled from $1bn to $2bn.
The futuristic design of the stadium also drew attention, with architects have likening it to a turtle or a bicycle helmet.