The most important rule concerning data collection and analysis is do not attempt to collect or analyze all possible kinds of data . Unless you are conducting a truly explorative study (which is hardly ever necessary nowadays, considering the abundance of literature on most topics), the first part of the empirical cycle—the process of theory development—should result in clear research questions or hypotheses that will allow you to choose an appropriate design to study these. These hypotheses should also indicate the kind of data you will need to collect—that is, the data you have hypotheses about and some necessary control data (e.g., time on task), and
together with the design provide some indications as to how to analyze those data (e.g., 2 × 2 factorial design,2 × 2 MANCOVA). But, these are just indications, and many decisions remain to be made. To name just a few issues regarding data collection (for an elaboration on those questions, see, for example, Christensen, 2006; Sapsford and Jupp, 1996): Which participants
(human/nonhuman, age, educational background, gender) and how many to use? What and how many tasks or stimuli to present and on what apparatus? What (control) measures to take? What instructions to give? What procedure to use? When to schedule the sessions?