Agriculture in mountainous areas in Vietnam has much intensified since the introduction of market-based reforms in the mid 1980s. The adoption of hybrid maize varieties, mineral fertilizers and reduction in fallow periods has improved farm incomes, but has also led to a dramatic increase in soil erosion from sloping lands which has created a downward pressure on crop yields and has had adverse effects on downstream areas. This study explores the relationship between soil fertility, crop yields and the use of soil conservation methods by applying an agent-based modeling approach that combines whole-farm mathematical programming to simulate the decision-making of each individual farm household with a biophysical simulator of crop yields and soil fertility dynamics for each individual landscape unit. Simulation results suggest an average soil loss is 30 tons for maize fields and 27 tons for cassava fields per hectare per annum under present economic conditions, which is in the range of what other studies have measured, and a consequent decline in the average household incomes by 28.5% over a 25 years period. The introduction of three soil conservation methods in maize (vetiver grass strips, ruzi grass barriers and leucaena hedges) shows that these are not economical for farm households to adopt under present conditions, chiefly because of lower short-term maize yields. We explore the effect of giving farm households monetary incentives to adopt soil conservation and find that the payment needed for reducing 40 ± 2% of the estimated soil loss would be about 12–16 USD per ton of soil saved.