Issues in the Use of Expert Judgement
Selection of Experts
Wide enough to encompass all facets of scientific thought on the topic
Qualificationscriteria need to be specified
Pitfalls in Elicitation – Biases
Mindsets – unstated assumptions that the expert uses
Structural Biases – from level of detail or choice of background scales for quantification
Motivational Biases – expert has a stake in the study outcome
Cognitive Biases
Overconfidence – manifested in uncertainty estimation
Anchoring – expert subconsciously bases his judgement on some previously given estimate
Availability – when events that are easily (difficult) to recall are likely to be overestimated (underestimated)