What is a Pilot Study?
Ashley is a graduate student who wanted to conduct a research study to test whether small group math games helped children learn more math. She designed an experiment to research this question, which included three teachers facilitating small group math games with children. It was important to pick the right games and establish exactly how the games would be facilitated so that each teacher was doing exactly the same thing. These were good reasons to conduct a pilot study before the main study.
A pilot study is a research study conducted before the intended study. Pilot studies are usually executed as planned for the intended study, but on a smaller scale. Although a pilot study cannot eliminate all systematic errors or unexpected problems, it reduces the likelihood of making a Type I or Type II error. Both types of errors make the main study a waste of effort, time, and money.
Reasons to Employ a Pilot Study
There are many reasons to employ a pilot study before implementing the main study. Here are a few good reasons:
To test the research process and/or protocol. These are often referred to as feasibility studies because the pilot study tests how possible the design is in reality. For example, are the study resources adequate, including time, finances, materials? Are there are any other logistical problems that need to be addressed?
To identify variables of interest and decide how to operationalize each one. For instance, what are the indicators of composite variables? How will variables be measured and/or computed?
To test an intervention strategy and identify the components that are most important to the facilitation of the intervention.
To test methodological changes to implementation or administration of an instrument and/or train personnel on the administration of instruments.
To develop or test the efficacy of research instruments and protocols. Are there confusing or misleading questions? Is it possible to maintain maximum objectivity and reduce observer drift?
To estimate statistical parameters for later analyses. Certain statistical analyses require the sample size is sufficiently large and contains enough variability to detect differences between groups, given there any real differences to be detected.
For Ashley's graduate research study, she conducted a pilot study to see what small group math game would work best, including how long it would take to play the game. She also established the protocol for facilitation and then practiced that protocol with a large number of children. Although the pilot study helped perfect the implementation of the main study, there were still limitations to consider.
Limitations
Some of the most formidable problems in a large scale study come from the size of the study, including the number of participants, the number of staff required to implement the study, and the amount of data that must be maintained, organized, and entered into computer programs. Many of these problems associated with the larger scale may not be evident in a smaller pilot study and must be managed as they arise in the main study.
To un