Let’s think now about how runaways might work in a social system like a business. Certainly
we humans have the ability to create incentives for bad choices that do not contribute to the
long-term health of our enterprises. Any manager who has had to design a compensation
scheme knows this; as often as not, bonuses wind up rewarding behavior contrary to the
organization’s espoused mission and values. (Steven Kerr summed up this problem nicely in
his classic article, “On the Folly of Rewarding A, While Hoping for B.”) The problem is
reinforced when large bonuses result in prestige for individuals, rather than in increases in
some harder-to-trace sense of overall value. The more this feedback loop self-reinforces, the
harder it is to change.
In most cases, whether in nature or in man-made systems, misalignments are easy to spot and
don’t persist for long. The more insidious problems arise when the proxy for the health of the
system starts out valid but then becomes increasingly obsolete as conditions change—and no
one tells the peahens, whose pecking order has grown to depend on having the mate with the
largest tail.