Taylor, one of many boxing champions hailing from the city of Philadelphia, Pennsylvania, learned his craft in the gyms of his hometown and posted a 99-4 record as an amateur fighter. In 1984, Taylor earned a spot on the 1984 U.S. Olympic team at the age of 17, and claimed the gold medal in the featherweight division. Following his victory, he joined the professional ranks.