Regionwide Implementation
To expand the successful pilot campaign to the entire six-state region, several changes and new procedures had to be put in place. Each state's campaign was directed at the state level, with an on-site campaign coordinator at each participating clinic. The materials were customized for each state with its own toll-free phone number. Based on the pilot testing, the contractor knew to leave extra time for printing and dissemination of the materials to make sure they were out in the community at the same time as the radio spots were airing. In addition, the posters were made smaller and only two of the four designs were reprinted. Different sets of the radio spots were used in each state, based on the results of the pilot test and the decision of the state-level coordinator as to which spots were most appropriate for her state.
The campaign was implemented in 55 cities throughout the six states for a three-month period. An independent evaluator was hired to assess the effectiveness of the regionwide campaign. The process evaluation consisted of: 1) Qualitative questionnaires completed by the state grant directors, clinic managers in targeted cities and the project director; and 2) Focus groups with clinic staff in three sites. The outcome evaluation provided qualitative and quantitative data from 19 evaluation sites regionwide through: 1) Pre- and post-campaign KAB surveys; 2) Qualitative in-depth interviews with members of the target audience; and 3) Caller tracking sheets from the clinic evaluation sites.
Process Evaluation
The process evaluation identified some areas that required improvement. Despite having begun production of the print materials well in advance, one state delayed printing because it took much longer than expected to receive approval through its bureaucratic channels for a toll-free phone number. In addition, a mix-up by the printer resulted in boxes of coasters being sent to the wrong states. This cut the time available for the state-level coordinators to get the materials out to the local clinics and delayed the start date of the campaign.
Although a comprehensive implementation guide was provided to each state-level coordinator to assist clinics in their dissemination efforts, some clinic managers did not always read it or required additional guidance. This resulted in several local clinics not distributing the print materials _ such as posters, brochures and coasters _ in their communities. There were clear differences in the visibility of the campaign in cities with active clinic outreach efforts versus those that relied solely on the radio spots and newspaper ads.
The focus groups with clinic staff revealed that some of the clinics actually taped over the toll-free phone numbers on the posters and wrote in their own local numbers. Several participants noted that their clients would feel more secure knowing exactly who they were calling. In addition, staff at participating clinics preferred to use the publicity from the campaign to increase clients at their own clinics versus other competing providers. The low usage of the toll-free numbers also suggested that use of local phone numbers might be preferable.
When buying newspaper space, a small number of the media outlets declined to run the ads because their staff felt they were too explicit. This occurred in some of the college newspapers _ particularly the religiously-affiliated schools _ and in community newspapers primarily in South Dakota and Utah. Two major newspapers in South Dakota refused to run the ads, but then both wrote substantial articles about the campaign that included the main messages and toll-free number, with one even including a picture of one of the posters.
Outcome Evaluation
The outcome evaluation produced both good news and bad news. The bad news was that the campaign media had not saturated the markets to the extent hoped for: Based on the post-campaign KAB survey, only 15 percent of the target audience reported being exposed to the "Don't Kid Yourself" campaign. The good news was that those target audience members who were exposed to the campaign responded quite positively.
The survey revealed that 92 percent of those exposed to the campaign materials reported that they "liked the message." Nearly 70 percent saw a poster, more than double the exposure of any other campaign medium. Significant attitude changes about the use of birth control were also correlated to the number of campaign media to which the respondent was exposed.
Survey respondents who were exposed to the campaign also overwhelmingly took some desirable action. More than three-quarters of those exposed indicated that they "talked with friends, boy/girlfriend, or parents about family planning" as a result of the campaign. Over 55 percent reported that they called for information or an appointment at a doctor's office or family planning clinic as a result of the campaign, but fewer than 5 percent of the respondents called the toll-free phone number advertised in the materials. Despite not calling the toll-free number, about a quarter of the respondents said that they called for an appointment at a family planning clinic as a result of the campaign _ most likely on the clinic's local phone number.
The campaign started its third phase of implementation in Spring 1998. Based on results of the first year of regionwide implementation, the next iteration of the campaign focused on a smaller number of cities in each state to maximize the media exposure that was possible with available resources. The use of toll-free numbers was discontinued in most states, and materials were created for each community using local clinic names and phone numbers. Greater outreach and accountability from local clinics was required as a condition of the inclusion of their community in the campaign.