Sometimes numerical measures of political phenomena such as policy impacts are used without sufficient care in determining their validity. Is the number of infant deaths (in their first year) per 1,000 live births a good indicator of the general level of health care in a society that has much income inequality? Do salary levels and similar data really measure the professionalism of civil servant? The acquisition of hard facts about who did what, why, and with what effect should be the goal of research. We need to be able to say with some certainty why members of Congress respond to constituency interests on some issues and not others, or what role media play in setting agendas.
In explaining behavior in the policy process, one needs empirical data that will permit the demonstration or sound inference of cause-and-effect relation-ships. Once a person gets involved in quantitative data-based analysis, it is important to resist the notion that collecting empirical data is of prime importance and that the more data one has, the more one can explain. One can drown in a sea of data as well as thirst for lack thereof. To account for or explain behavior, theory is needed that will guide analysis in potentially fruitful directions, as well as good judgment in the selection of policy measures. As much as possible, hypotheses about cause-and-effect relationships need to be developed and tested on the basis of the best available evidence.
The notion that policy analysis is worthwhile only when it involves the analysis of quantitative data with statistical techniques--the higher powered the better--should also be resisted. There is no reason to assume that if something cannot be counted, it does not count. Some policy areas and problems have not been very amenable to rigorous quantitative measurement and analysis, although this may not always continue to be the case. Many aspects of social welfare and economic regulatory policies currently fit into this category. How does one measure the comparative influence of pressure groups, agency values, and economic analysis on rulemaking by the EPA or OSHA? The prosecution of insider traders by the Securities and Exchange Commission? The total benefits of a public-housing program? And how does one appraise the power of ideas, as distinct from interests, in developing programs for the handicapped? Such questions present real puzzles.