What New South Wales police said may have been a bull shark was sighted 150 metres (yards) from the shore by a police helicopter off the popular eastern suburbs beach.
"We've had a lot of fish in the bay and that's part of their (sharks) general routine, they eat fish -- sharks, and so they're following the bait ball," Bondi lifeguard Anthony Carroll told the Australian Broadcasting Corporation.
A bait ball is a densely packed school of fish that forms when they are threatened by predators.
"The bait ball went up into the northern end of Bondi and then went back out to sea, so after an hour we've put the flags back up here and we've all got all the swimmers back out in the water.
"There's so much fish for the sharks to eat I wasn't so concerned about the swimmers, but part of our policy is to pull the flags out and shoo the shark out to sea."
The beach was reopened after just under two hours, a local council spokeswoman told AFP.
Sydney beaches have attracted thousands over the summer holidays amid several days of hot weather.
Swimmers were also ordered out of the water at Bondi in November after a shark sighting, which followed the discovery of the carcass of a great white pulled from nets set up offshore to protected bathers.
Beaches across Sydney and New South Wales state have been partially netted since 1937.
The netting does not stretch the entire length of Bondi and is meant to create a barrier between swimmers and sharks. It is also designed to stop the animals establishing territories where people use the water.
A 17-year-old boy died earlier this week after he was bitten on the leg by a shark while spear-fishing off a beach in southwestern Australia.
It was the second fatal mauling in Australia since December 15, when a teenager was attacked while swimming off Port Douglas in the northeast.
Experts say attacks by sharks are increasing as water sports become more popular.