The 22-year-old was in the water at Bombo Beach, 117 kilometres (73 miles) south of Sydney, on Wednesday evening when the shark attacked.
"The man has told police he was attacked in the water about 100 metres offshore but did not see the shark," a New South Wales state police spokesman said.
"He was assisted to the beach by a friend and emergency services were contacted."
The Sydney Morning Herald said the man had three-quarters of his thigh ripped off, as well as hand injuries as he fought the shark, with his screams alerting fellow surfers and bystanders.
He was helped ashore by a friend with two off-duty nurses on the beach applying a tourniquet around his bloody thigh that helped save his life.
"He had lost a large proportion of his left thigh, and the quad muscle was torn away right down to the bone," Terry Morrow from Ambulance NSW told reporters.
"He could have bled to death before we arrived on scene. He was very lucky the members of the public were there and acted as they did. They saved his life, to tell you the truth."
Shark expert Michael Brown from Surf Watch Australia said the culprit was likely either a great white or bronze whaler.
"If we look at the horrific injury, it's probably fairly obvious it's a great white or a bronze whaler," he told the Seven Network.
A spate of attacks in New South Wales state last year led authorities to implement a range of shark-prevention measures to reduce encounters.
Measures include the testing of aerial drones to track sharks' movements, although the state government has ruled out culling the animals.
There were 14 attacks in New South Wales last year -- including the death of a Japanese surfer -- compared to three in 2014, according to data compiled by Sydney's Taronga Zoo.
Experts say attacks are increasing as water sports become more popular and bait fish move closer to shore, but fatalities remain rare.