Until recently, program evaluators tended to consider technical quality and methodological rigor their primary concerns
But Patton’s (1986) introduction of ‘utilization focused evaluation’ challenged this scientific attitude
In utilization focused evaluation utility becomes the critical criterion for evaluation excellence although the issues of validity reliability measurability and generalizability remain important
In empowerment evaluation with a high utility face validity holds great value
Evaluation participants apply ‘expert judgment’ to appraise the face validity of the measurement
If its predictive validityconcurrent validity and construct validity also prove to be high the instrument acquires additional power
The Community Living Skills Scale (Smith and Fordม 1990) a product of empowerment evaluation as the example shows*represents an instrument with a reasonably high discriminant and concurrent validity as well as acceptable reliability
Despite the virtues just described empowerment evaluation with consumer involvement poses a few methodological problems including a possible discrepancy between predetermined program objectives and the objectives the staff members or consumers specify
For example when the mental health consumer group began to develop a functional level scale some group members disapproved of the program objective of improving the consumers’ daily living skills and insisted that the primary function of the program should be to improve their emotional state
The evaluator stepped in as a facilitator reminding the members that mental health service consumers should acquire or relearn community living skills if they seek to remain in the community then suggesting
that the agency recognize improvement in emotional state as an equally important objective
Thus, the evaluator’s role as a trainer facilitator and advocate proved critical in ensuring the integrity of the evaluation design and to play those multiple roles an external evaluator must be well grounded in the agency’s operation