Chapters 8 through 11 have dealt with several different aspects of survey research. In the course of outlining procedures associated with the survey, certain problems with the techniques with which it is typically associated have been identified. To some extent the deficiencies associated with the survey are recognized by practitioners, who have developed ways of dealing with them or at least of offsetting their impact to some degree. When survey techniques such as the structured interview or the self-completion questionnaire are employed in connection with the study of respondents’ behaviour, certain characteristic difficulties are encountered, some of which have been touched on in earlier chapters. Tips and skills ‘Problems with using survey research to investigate behaviour’ identifies some of the difficulties entailed in using survey methods to research behaviour. The list is by no means exhaustive but it does capture some of the main elements.
Tips and skills
Problems with using survey research to investigate behaviour
• Problem of meaning. People may vary in their interpretations of key terms in a question (see Thinking deeply 12.1).
• Problem of omission. When answering the question, respondents may inadvertently omit key terms in the question (see Thinking deeply 12.1).
• Problem of memory. They may misremember aspects of the occurrence of certain forms of behaviour.
• Social desirability effect. They may exhibit a tendency towards replying in ways that are meant to be consistent with their perceptions of the desirability of certain kinds of answer.
• Question threat. Some questions may appear threatening and result in a failure to provide an honest reply.
• Interviewer characteristics. Aspects of the interviewer may influence the answers provided.
• Gap between stated and actual behaviour. How people say they are likely to behave and hoe they actually behave may be inconsistent (see Thinking deeply 12.2).
Thinking deeply 12.1
Accurate reporting of behaviour and the problems of meaning and omission
Belson (1981) has conducted detailed studies of how people interpret questions designed to gauge attitudes and behaviour. One question concerned with the latter was embedded in a structured interview schedule administered to fifty-nine British adults and went as follows:
When you turn on your televisions in the evening, do you generally go on viewing till the end of the evening or do you just watch one or two programmes? (Belson 1981:59)
Intensive interviews undertaken after the structured interviews had been carried out revealed that on respondents interpreted the question totally correctly. Twenty-five respondents arrived at incorrect interpretations; the rest were broadly correct but in varying degrees. A common problem was that the question was designed to refer to when the respondents themselves turned the television on. This was correctly interpreted by thirty-eight respondents, but fifteen interpreted the question to mean when the set was switched on---that is, not necessarily by the respondent (problem of meaning). Nine respondents appeared not to have taken any notice of the phrase ‘when you turn on your television’ (problem of omission). Similarly, ten failed to consider ‘till the end of the evening’ in their answers, while ‘generally’ spawned several interpretations. We see here problems of omission and meaning respectively.
Thinking deeply 12.2
Gap between stated and actual behaviour
This is one of the most infamous cases of problems of the gap between what people say they do (or are likely to do) and their actual behaviour. Questionnaires tap people’s attitudes and reports of their behaviour, but one might legitimately question how well these relate to actual behaviour. A study of racial prejudice conducted many years ago by LaPiere (1934) illustrates this issue. He spent two years travelling with a young Chinese student and his wife to determine whether they were refused entry at hotels and restaurants. They twice crossed the USA. Of 66 hotels, they were refused entry once; of 184 restaurants and dinners, none refused entry. LaPiere sought to eliminate himself as a possible contaminating influence by ensuring that he was not involved in gaining access to the various establishments and indeed seems to have sought to load the dice slightly in favour of being turned away;
Whenever possible I let my Chinese friend negotiate for accommodation . . . or sent them into the restaurant ahead of me. In this way I attempted to ‘factor’ myself out. We sometimes patronized high-class establishments after a hard and dusty day on the road and stopped at inferior auto camps when in our most presentable condition. (LaPiere 1934: 232)
Lapiere then allowed six months to elapse and sent questionnaires to the hotels and restaurants they had visited. One of the question asked: ‘Will you accept members of the Chinese race as guests in your establishment?’ Of the establishments that replied, 92 per cent of restaurants said no; and 91 per cent of hotels said no. LaPiere’s simple though striking study clearly illustrates the gap that may exist between reports of behaviour and actual behaviour. It should also be noted that the question asked is somewhat unclear; a feature that is not usually remarked upon in connection with this widely cited study. ‘Will you . . . ?’ can be interpreted as asking the respondent to project into the future or to state the establishment’s policy. Quite why the more obvious formulation of ‘Do you . . . ?’ was not used is not clear, though it is unlikely that this point has a significant bearing on the finding and their implications for survey research.