Two psychologists (judges) evaluate 50 patients as to whether they are psychotic, borderline or neither. The results are summarized in Figure 1.
We use Cohen’s kappa to measure the reliability of the diagnosis by measuring the agreement between the two judges, subtracting out agreement due to chance, as shown in Figure 2.