outcomes. Three process measures were
tracked—the importance of project goals,
member satisfaction and project implementation
(Table 1).
•Importance of project goals. To build consensus
on and to set priorities for changes
outlined in the project action plan, community
members involved in the initiative
completed a paper-and-pencil survey
about the project goals.15 Those surveyed
were community members who had participated
in project activities or who had
served on the project board or a committee,
and individuals who project staff wanted
to keep informed or who were supportive
of the project’s goals. The survey
listed the potential community and systems
changes to be sought by the project.
Members were asked to rate the importance
of each proposed change as it related
to the mission of the initiative—reducing
adolescent pregnancy. A five-point
Likert scale (with one representing “very
unimportant” and five “very important”)
was used to rate each proposed change.
•Community member satisfaction. Researchers
used a mailed member satisfaction
survey to assess community members’satisfaction
with the development
and functioning of the projects.16 The survey
contained specific questions related
to the day-to-day functioning of the project,
including leadership, planning, services,
community involvement and
progress toward accomplishing project
goals. Community members rated their
satisfaction using a five-point Likert scale
(with one representing “very unsatisfied”
and five “very satisfied”).
•Project implementation. Several measures
were associated with project implementation
and replication, including community
actions, media coverage, services
provided, community health education,
sexuality education provided to students
and teachers, and resources generated. A
monitoring and feedback system was
used to track these measures.17 Project staff
reported events and services to evaluators
on monthly log forms, which were then
coded and summarized. Evaluators clarified
log entries by conducting semistructured
interviews with project staff.
Archival records, such as meeting minutes
and newspaper articles, were also used to
help verify log entries. For 95% of instances
of community services and
changes, there was an interobserver agreement
of 91.4% (Kappa = .893).
We also measured three intermediate
outcomes of the initiative—community
change, the importance of intermediate
outcomes and critical events.
•Community change. Community and systems
changes (new or modified programs,
policies or practices that were consistent
with the mission) were tracked to assess
implementation of interventions that
might reduce the risk of adolescent pregnancy.
Project staff completed and sent
monthly event logs to evaluators.18 Evaluators
coded log entries and summarized
and graphed the data. Semistructured interviews
and archival records (e.g., meeting
minutes) were used to clarify and verify
information reported in the logs.
•Importance of intermediate outcomes. Community
members and experts in the field
of teenage pregnancy used a paper-andpencil
survey of outcomes to assess the importance
of community changes facilitated
by the project.19 Evaluators created a
survey listing each community change that
had been measured using the monitoring
system. Respondents rated the importance
of each community change in reducing the
risk for adolescent pregnancy.