Localized surface plasmon resonance (LSPR) combined with immunoassay shows greatly potential in fast detection of tumor markers. In this paper, a highly sensitive LSPR substrate has been fabricated and modified for direct detection of alpha-fetoprotein (AFP). The biosensor was prepared by interference lithography, and modified by covalently immobilizing anti-AFP on the surface of gold nano-mushroom arrays (GNMA). The modification process was investigated by Vis–NIR reflectance spectra and cyclic voltammogram measurements. We revealed the optical properties of the modified GNMA by measuring the Vis–NIR reflectance spectra and simulating its electric intensity field distribution under light illumination. The GNMA substrate was highly sensitive, with a refractive index sensitivity of ~465 nm/RIU. The substrate can be applied to label-free detection of AFP, with the linear range and the limit of detection determined to be 20–200 ng/mL and 24 ng/mL (S/N=3), respectively. We also demonstrated its clinical application by directly detecting AFP in human serum samples. It is expected that our biosensor could be integrated on microfluidic chips for high-throughput detection in portable early diagnosis, post-operative and point-of-care (POC) in clinical applications.