7. Following up surprises Surprises can be salutary. You may well be surprised because something is at variance with your (possibly implicit and not thought through) theory of what is going on. This then provides the opportunity to bring that theory to the surface, possibly to revise it, and to search for evidence relevant to the revision.
8. Looking for negative evidence This is the tactic of actively seeking disconfirmation of what you think is true. While this is in principle straightforward, you are likely to have some reluctance to spending a large amount of effort on this activity. Miles and Huberman (1994, p. 271) make the helpful suggestion of giving a colleague your conclusions and free access to your original data with a brief to try to find evidence which would disconfirm your conclusion. If they manage to do this then your task is to come up with an alternative, broadened, or elaborated explanation.
Testing explanations
9. Making if-then tests Testing possible relationships: i.e. if one condition obtains or is the case, look to see if a second one is. If it is, we are on the way to understanding what is going on and can make further similar tests. If it isn’t true we have to make other conjectures.
10. Ruling out spurious relationships If you appear to have established a relationship, consider whether there may be a third factor or variable which underlies, influences or causes the apparent relationship. In the relationship between guardsmen fainting on parade and the softness of the tar in the asphalt of the parade ground, it appears highly likely that the temperature is providing an intervening causal link rather than noxious fumes from the tar having a direct effect. Note that this is essentially the same tactic discussed above under the heading of ‘finding intervening variables’ but used for a different purpose. It can also be thought of as finding rival explanations for a relationship.
11. Replicating a finding If a finding can be repeated in a different context or data set, then it is more dependable. Given that once you find a relationship or develop a theory there is a strong tendency for you to find confirming evidence (and to ignore disconfirming evidence), it is even better if someone else, not privy to your findings, confirms it. Note that this is a particular type of triangulation.
12. Checking out rival explanations It is good practice to try to come up with one or more rival explanations which could account for all or part of the phenomena you are studying. Keeping these ‘in play’ while you are analyzing and gathering further data helps-to prevent the premature closure effect discussed above.
13. Getting feedback from informants This process of ‘member checking’ performs several useful functions. It honours the implicit (or preferably explicit) contract between researcher and informant to provide feedback about the findings. It also provides an invaluable means of corroborating them. While problems of jargon and terminology may need to be attended to, you should be able to present findings in a way that communicates with informants and allows them to evaluate the findings in the light of their superior experience of the setting.