Coding Consistency Checks
Independent parallel coding. An initial coder carries out an analysis and develops a set of
categories that constitute the preliminary findings. A second coder is given the evaluation
objectives and some or all of the raw text from which the initial categories were developed.
Without seeing the initial categories, the second coder is asked to create a second set of categories
from the raw text. This second set of categories is then compared with the first set to
establish the extent of overlap. The two sets of categories may subsequently be merged into a
combined set. When overlap between the categories is low, further analysis and discussion may
be needed to develop a more robust set of categories.
Check on the clarity of categories. An initial coding of the raw data is completed. A second
coder is given the evaluation objectives, the categories developed, and descriptions of each category,
without the rawtext attached. The second coder is then given a sample of the rawtext (previously
coded by the initial coder) and asked to assign sections of the text to the categories that
have been developed. A check can then be made of the extent to which the second coder allocated
the same text segments to the initial categories as the first coder. A variation of the check
on the clarity of the categories is to give a second independent coder both the initial categories
and some of the text assigned to these categories. The second coder is then given a new set
of text that has not been coded and asked to assign sections of the new text into the initial
categories.