Fang Xuanling was born in 579, shortly before the founding of the Sui dynasty in 581, during Sui's predecessor state, Northern Zhou. His great‐grandfather Fang Yi (房翼) was a general, official, and hereditary count under the Northern Wei dynasty, and his grandfather Fang Xiong (房熊) was also an official. His father Fang Yanqian (房彥謙) was a county magistrate during the Sui dynasty.
Fang Xuanling was said to be intelligent and well‐learned in his youth, and particularly skillful at calligraphy. It was said that once, when he accompanied his father to the capital Chang'an, the state was peaceful, and the popular sentiment was that Sui would last a long time. However, Fang Xuanling secretly opined to his father:
“
The emperor [i.e., Emperor Wen of Sui] had no accomplishments or virtues, and he gained his power by trickery. He did not think about long‐term benefits for his descendants, and he allowed his sons to have no differences between the heir and the non‐heirs. They therefore compete with each other and plot against each other, and they also compete in their wastefulness and luxuries. They will eventually kill each other, and the state will not be preserved. Even though everything is peaceful now, I expect its fall to be soon.
”
Fang Yanqian was surprised by his son's opinion, which, however, eventually turned out to be prophetic. When he was 17, he was successful at the imperial examination, and he became a military officer. He particularly impressed the deputy minister of civil service, Gao Xiaoji (高孝基). However, it appeared that he did not serve long, as his father became ill, and the illness lasted 10 years, during which Fang Xuanling attended to him earnestly. After his father's death, he fasted for five days. He later became the magistrate of Xicheng County (隰城, in modern Lüliang, Shanxi).
In 617, when the general Li Yuan rebelled against the rule of Emperor Wen's son Emperor Yang, one of Li Yuan's major generals was his son Li Shimin. Fang Xuanling went to Li Shimin's camp and offered his services. It was said that as soon as Li Shimin met Fang, they became like old friends, and Li Shimin invited him to serve on staff. Fang served Li Shimin faithfully, and wherever Li Shimin campaigned, while his staff members would collect treasures, Fang spent the time interviewing the people of the area and retained the capable people to add to Li Shimin's staff.
In 618, after news arrived at Chang'an (which Li Yuan had taken in winter 617 and where he declared Emperor Yang's grandson Yang You the Prince of Dai emperor (as Emperor Gong)) that Emperor Yang had been killed in a coup at Jiangdu (江都, in modern Yangzhou, Jiangsu), led by the general Yuwen Huaji, Li Yuan had Emperor Gong yield the throne to him, establishing the Tang dynasty as its Emperor Gaozu. He created Li Shimin, the Prince of Qin. Fang continued to serve on Li Shimin's staff.