The hypothesis of this study was that cumulative P fertilization decreases the contribution of arbuscular mycorrhiza (AM) to crop growth and nutrient uptake in Northern European field conditions. The modes of action of P fertilization were evaluated through effects on mycorrhization, crop dependence on AM, and AM fungal (AMF) community. Field studies were carried out within long-term experiments on soils with low and intermediate initial content of extractable P, where no P fertilization and 45 kg ha−1 a−1 P were applied for 20 years. AM effectiveness in terms of growth and nutrient uptake of flax, red clover and barley, percentage root length colonized by AMF, P response of flax, and spore densities and species composition of the AMF communities, were assessed. In the soil with low initial P supply, cumulative P fertilization decreased AM contribution to crop growth and nutrient uptake. The higher AM effectiveness in soil with no added P compensated the cumulative P fertilization (soil PH2O 2.5 v. 9.5 mg kg−1) for flax, but not completely for clover. In contrast, barley obtained no benefit from AM at harvest and only a slight benefit from cumulated P. In the soil with intermediate initial P supply, AM reduced growth of flax and barley, especially with no added P, and no response to AM was obtained on clover due to retarded mycorrhization. Cumulative P fertilization reduced yield losses of flax by AM (PH2O 18.8 v. 5.4 mg kg−1), because fertilization inhibited mycorrhization. In both soils, root colonization and spore density were decreased by cumulative P fertilization, but no changes in AMF species composition were observed.