Although it is not extensively used in social research, structured observation has been quite controversial. Certain criticisms have been implied in some of the previous discussion of reliability and validity issues, as well as in connection with the issue of generalizability. However, certain other areas of criticism warrant further discussion.
• There is a risk of imposing a potentially inappropriate or irrelevant framework on the setting being observed. This point is similar to the problem of the closed question in questionnaires. This risk is especially great if the setting is one about which little is known. One solution is for the structured observation to be preceded by a period of unstructured observation, so that appropriate variables and categorise can be specified.
• Because it concentrates upon directly observable behaviour, structured observation is rarely able to get at intentions behind behaviour. Sometimes, when intentions are of concern, they are imputed by observers. Thus, in the FIAC scheme (see figure 12.1), the category ‘teacher praises or encourages’ means imputing a motive to something that the teacher says. Similarly, Blatchford et al. (2009: 668) report that one of the categories of observation of pupil behaviour they employed in their study of the impact of teaching assistants on engagement in class was: ‘Individual off-task (passive): target child is disengaged during task activity, for example, day dreaming’. Essentially, the problem is that structured observation does not readily allow the observer to get a group of the meaning of behaviour
• There is a tendency for structured observation to generate lots of fragments of data. The problem here can be one of trying to piece them together to produce an overall picture, or one of trying to find general themes that link the fragments of data together. It becomes difficult, in order words, to see a bigger picture that lies behind the segments of behaviour that structured observation typically uncovers. It has been suggested, for example, that the tendency for structured observation studies of managers at work to find little evidence of planning in their everyday work (e.g. Mintzberg 1973) is due to the tendency for the method to fragment a manager’s activities into discrete parts. As a result, something like planning, which may be an element in may managerial activities, becomes obscured from view (Synder and Glueck 1980)
• It is often suggested that structured observation neglects the context within which behaviour takes place. Delamont and Hamilton (1984), for example, note in connection with the ORACLE research that it was found that teachers’ styles were related to their ages. However, they argue that such a finding can really be understood only ‘if data are gathered on teacher careers and life histories of kind eschewed by ORACLE’ (1984: 9). Of course, were such data collected, this criticism would have little weight, but the tendency of structured observation researchers to concentrate on overt behaviour tends to engender this kind of criticism.
On the other hand . . .
It is clear from the previous section that there are undeniable limitations to structured observation. However, it also has to be remembered that, when overt behaviour is the focus of analysis and perhaps issues of meaning are less salient, structured observation is almost certainly more accurate and effective than getting people to report on their behaviour though questionnaires. Also, although the point was made in the previous section that the observation of behaviour often necessitates imputing meaning to it, that is not to say that imputing meaning is always involved. With most of the categories of behaviour used by Blatchford et al. (2009), little of any assignment of motive is required. Also if video evidence is accumulated, the researcher is afforded the opportunity to review the evidence at length and not rush to a possibly snap decision about what is being observed. For example, Sampson and Raudenbush (1999) took video footage of Chicago streets to develop a measure of social disorder that included such indicators as: alcohol consumption in public; sale of drugs; street prostitution; and fight between adults or hostile arguing.
It may also be that structured observation is a method that works best when accompanied by other methods. Since it can rarely provide reasons for observed patterns of behaviour, if it is accompanied by another method that can probe reasons, it is of greater utility. Delamont (1976) in her research in a school found FIAC to be useful as a means of exploring differences in teaching style between teachers. However, she was able to get at some of the reasons for the quantitative differences that she discerned only because she had carried out some participant observation and semi-structured interviewing (two of main methods of qualitative research) in various school classes. For example, she compared two Latin teachers who were similar in certain respects but differed in terms of ‘the proportion of questioning to lecturing in their speech’ (Delamont 1976: 108). These differences in teaching style reflected contrasting views about teaching and differences in personal demeanour. Blatchford (2005) reports that the structured observation data that were collected in the research reported in Research in focus 12.1 were part of a wider study of the impact of variations in class size. The other methods employed were: termly questionnaire administered to teachers to guage their estimates of how they allocated time in classrooms between different activities; end-of-year questionnaires administered to teacher asking them about their experiences of the impact of class size; and case studies of small and large classes comprising some semi-structured observation of event and semi-structured interviews with teachers and the head teacher.
In laboratory experiments in fields like social psychology, observation with varying degrees of structured is quite commonplace, but in social research structured observation has not been frequently used. Perhaps one major reason is that, although interviews and questionnaires are limited in terms of their capacity to tap behaviour accurately, as noted above, they do offer the opportunity to reveal information about both behaviour and attitudes and social backgrounds. In other words, they are more flexible and offer the prospect of being able to uncover a variety of correlates of behaviour (albeit reported behaviour), such as social background factors. They can also ask questions about attitudes and investigate explanations that people proffer for their behaviour. As a result, researchers using questionnaire are able to gain information about some factors that may lie behind the patterns of behaviour they uncover. Also, not all forms of behaviour are liable to be accessible to structured observation and it is likely that survey research or researcher-driven diaries (see Key concept 10.1) are the only likely means of gaining access to them. However, greater use of structured observation may result in greater facility with the method, so that reliable measures of the kind developed in areas like education might emerge.