Questionnaire 4: ranking the chosen factors.
This phase of the procedure will involve each panel separately ranking the factors on each of their distinct
pared-down lists. Each ranked list will reflect the
priority order for the specific panel. In this phase,
each expert will individually submit a rank ordering of
the items: one ordering for each of the two lists—
infrastructure and expediency. The questionnaire also
will ask experts to submit comments explaining or
justifying their rankings.
A review of the literature had not identified such an
explanation of rankings in any Delphi study, although
Schmidt suggests soliciting helpful comments. Rohrbaugh
compared the Delphi methodology with a group
decision method based on social judgment analysis
(SJA), which uses a formal graphical method to present
the reasoning behind a panelist’s decisions to the
other members of the panel. Although the results using
SJA were not significantly more accurate, the groups
reached a higher degree of consensus, since the members
understood one another’s reasoning. Based on
Rohrbaugh’s study, the panels should arrive at consensus
more quickly if provided with some sort of
feedback about the panelists’ reasonings.