Abstract—A framework is developed to explore the connection
between effective optimization algorithms and the problems they
are solving. A number of “no free lunch” (NFL) theorems are
presented which establish that for any algorithm, any elevated
performance over one class of problems is offset by performance
over another class. These theorems result in a geometric
interpretation of what it means for an algorithm to be well
suited to an optimization problem. Applications of the NFL
theorems to information-theoretic aspects of optimization and
benchmark measures of performance are also presented. Other
issues addressed include time-varying optimization problems and
a priori “head-to-head” minimax distinctions between optimization
algorithms, distinctions that result despite the NFL theorems’
enforcing of a type of uniformity over all algorithms.