Tips:
If a sample has several appraisers at 50%, that sample is probably marginally good or marginally bad. It may be a good sample to use to clarify training.
If a sample is totally misclassified by the majority of the appraisers, you may need to assess the standard and possible revise the inspection procedure and training to cover the issue that caused the consistent misclassification