Production of the silt-sized particles composing loess represents a critical step in its ultimate formation, but the volume and rate of production in different settings remain underdetermined. This study addresses the influence of climate and source lithology on silt formation by subjecting natural alluvial/fluvial sand from systems characterized by varying climatic setting to simulated comminution. Bulk samples from comparable transport distances were collected from a hot-arid (California desert) alluvial system, hot-humid (Puerto Rico) fluvial system, and cold-humid (Norway) proglacial fluvial system, and sieved to isolate the coarse-very coarse sand fraction for experimental milling. Source rocks in all three systems consist of granitoids, with some contributions from foliated coarse-grained metamorphic rocks (gneissic and schistose) in Norway. Results indicate that contributions from coarse-grained foliated lithologies impart a negligible effect on rate of silt production from the sand starting material. Climatic setting imparts no significant difference on sand durability, but influences clay production at later stages of comminution for the Puerto Rico sample, presumably owing to the effects of deep chemical weathering prior to physical transport. Comminution and silt production in all experiments exhibit exponential decay wherein rapid initial rates diminish with time. This is interpreted to reflect the effects of the dynamic milling conditions (particle size and distribution, and slurry viscosity), as well as particle strength, with the latter including both relative strengths of different minerals in general, and the influences of lattice defects and microfractures in quartz in particular. Empirical observations reveal that silt fractions of the natural alluvial/fluvial samples, collected at comparable transport distances (5–7 km) in the respective systems, exhibit significant differences. Bulk sediment samples from the Norwegian proglacial system contain substantially more silt (3–10 times) than either of the other two localities, implying a significant difference in the natural processes operating to generate silt in these different field settings. We suggest this difference reflects the efficacy of glacial grinding in silt production.