Eleven field experiments were conducted in Yongshou County, Shanxi Province and seven in Luoyang, Henan Province, and five layers of soil were sampled in 20-cm increments to 100 cm depth for each experimental field to study the difference of wheat (Triticum aestivum) responses to nitrate and ammonium nitrogen (N) and the relation of its responses to nitrate N cumulative amount in different soil layers. In Yongshou, each experimental site included seven treatments (control, two varieties of nitrate N, two varieties of ammonium N, and two combinations of ammonium to nitrate N in a ratio of 1:2) while in Luoyang, six treatments (only one variety of nitrate N). Total 150 kg ha−1 N was added as basal for N fertilization treatments at wheat seeding time. Results showed that differences in two varieties of each N form or combinations were much less than those of N forms. As a whole, wheat responses to nitrate N were better in most cases and in few cases, there was no significant difference between nitrate N and ammonium N on wheat yield. Using averages of yield, yield increase amount (kg ha−1) and percentage (%) of different N forms for comparison, nitrate N was highest in yield and yield increase, followed by the combination, while the ammonium N was lowest. The lower the amount of accumulated nitrate N in the soil profile, the better the nitrate N superiority. Our finding was that the superiority of nitrate N over ammonium N on wheat yield depended on the accumulative nitrate amounts in soil, and only in soil having low cumulative nitrate N amount at the rooting depth, did the nitrate N show its superiority over the ammonium N. In contrast, if the accumulative nitrate N amount was high, the two N forms had no significant difference on wheat yield.