Multiple measures are necessary to reflect multiple objectives and to avoid distorting performance. One can imagine schools developing and publishing a variety of measures of skill, knowledge, and satisfaction of students some immediate and some based on longer-term follow-up. One can imagine health programs developing a variety of measures of health status and satisfaction of patients also with different time lags. One can imagine manpower program developing a variety of measures of skills acquired and subsequent job success of trainees.
For some purposes measures without any weights would be sufficient. In a voucher system for education, for example, one could simply make available a variety of performance measures for each school and let parents and students choose among them according to their own weights systems. On the other hand, in a federal grant program designed to encourage effective manpower training, it would be necessary to assign weights to the various success measures being used. If several are being combined, as no one measures is allowed to dominate and distort the reward system.
Second, performance measures must reflect the difficulty of the problem. If absolute levels of performance are rewarded, then schools will select the brightest students, training programs will admit only the workers who will be easiest to place in jobs, health centers will turn away or neglect the hopelessly ill. To avoid these distortions, social service effectiveness must always be measured in relation to the difficulty of the task. In general, measures of change are better than measures of absolute level, but even this approach may not solve the problem. It may be easier to bring about significant changes in the performance of bright children than in that of retarded children or to improve the health status of certain classes of patients. In this situation, the success of a social action activity can be measured only in relation to success of other activities with the same kind of student or patient or trainee. A considerable period of time will be necessary to collect experience and delineate above- and below- average performance with particular types of problems.
None of this sounds easy to accomplish. And it isn’t. Nevertheless, we are unlikely to get improved social services (or, indeed, to know if we have them) until we make a sustained effort to develop performance measures suitable for judging and rewarding effectiveness. Current effort to publish test scores or infant mortality rates in the name of “assessment” or “accountability” are only the first halting steps on the long road to better social services.
Performance measures for social services are not, of course, ends in themselves. They are prerequisites to attempts both to find more effective methods of delivering social services and to construct incentives that will encourage their use. But all the strategies for finding better methods discussed in these pages, especially social experimentation, depend for their success on improving performance measures. So do all the models for better incentives. Put more simply, to do better, we must have a way of distinguishing better from worse.