Box 14.9
Assessing the quality of qualitative data analysis: tactics suggested by Miles and Huberman
Assessing data quality
1. Checking for representativeness There are many pitfalls to the gathering of representative data. The informants, and the events or activities sampled, may be non-representative. Safeguards include the use of random sampling where feasible; triangulation through multiple methods of data collection; constructing data display matrices; and seeking data for empty or weakly sampled cells. You analysis may be biased, not only because you are drawing inferences from non-representative processes, but also because of your own biases as an information processor (see p. 460). Auditing processes by colleagues help guard against this.
2. Checking for researcher effects These come in two versions: the effects you have on the case, and the effects your involvement with the case have on you. They have been discussed previously (p. 311).
3. Triangulation Again discussed earlier (p. 371). Not a panacea , and it has its own problems. (What, for example, do you do when two data sources are inconsistent or conflicting? Answer: you investigate further, possibly ending up with a more complex set of understandings.) However, it is very important: ‘triangulation is not so much a tactic as a way of life. If you self-consciously set out to collect and double-check findings, using multiple sources and modes of evidence, the verification process will largely be built into data collection as you go’ (Miles and Huberman, 1994, p. 267).
4. Weighting the evidence Some data are stronger than others and you naturally place greater reliance on conclusions based on the former. Stronger data are typically those you collect first-hand; which you have observed directly; which come from trusted informants; which are collected when the respondent is alone rather than in a group setting; and which arise from repeated contact.
Testing patterns
5. Checking the meaning of outliers These are the exceptions, the ones that don’t fit into the overall pattern of findings or lie at the extremes of a distribution. Outliers can be people, cases, settings, treatments or events. Don’t be tempted to hide or forget them.
6. Using extreme cases These are outliers of a particular type, defined in terms of being atypical situations or persons rather than by the data they provide, which may or may not be atypical. An innovation which failed in a school where the circumstances appeared close to ideal seemed to be linked to the unexpressed resistance of the deputy head teacher responsible for timetabling, hence suggesting a key factor.