Steps 1 to 3 are usually called pre-experimental planning. It is vital that these steps
be performed as well as possible if the experiment is to be successful. Coleman and
Montgomery (1993) discuss this in detail and offer more guidance in pre-experimental planning, including worksheets to assist the experimenter in obtaining and documenting the
required information. Section S 13.1 of the supplemental text material contains additional useful material on planning experiments.
Throughout this entire process, it is important to keep in mind that experimentation is
an important part of the learning process, where we tentatively formulate hypotheses about a
system, perform experiments to investigate these hypotheses, and on the basis of the results
formulate new hypotheses, and so on. This suggests that experimentation is iterative. It is
usually a major mistake to design a single, large comprehensive experiment at the start of a study. A successful experiment requires knowledge of the important factors, the ranges over
which these factors should be varied, the appropriate number of levels to use, and the proper
units of measurement for these variables. Generally, we do not know perfectly the answers to
these questions, but we learn about them as we go along. As an experimental program progresses, we often drop some variables, add others, change the region of exploration for some
factors, or add new response variables. Consequently, we usually experiment sequentially,
and as a general rule no more than about 25% of the available resources should be invested
in the first experiment. This will ensure that sufficient resources are available to accomplish
the final objective of the experiment.