The Appraisal of Guidelines for Research and Evaluation (AGREE) II
Instrument14 was used to assess the methodological quality of the
guidelines. The 23-item instrument has been internationally validated
and consists of six domains: scope and purpose, stakeholder
involvement, rigour of development, clarity and presentation, applicability
and editorial independence. A definition of these domains
can be found in the Table S2. Each guideline was independently
appraised by PLV and AT. Each item within the six domains was
rated by allocating a value from 1 to 7 (1 = ‘Strongly Disagree’;7 = ‘Strongly Agree’) based on the specific assessment criteria provided.
Major discrepancies in the scores were discussed and independently
reassessed. Domain scores were calculated as per the
AGREE II user’s manual, whereby a total quality score was obtained
for each domain by summing up the scores of each item.14 A
maximum possible score for each domain was calculated by multiplying
the number of appraisers by the number of items for that
domain and multiplying by seven (value for ‘strongly agree’). A
minimum possible score for each domain was calculated by multiplying
the number of appraisers by the number of items for that
domain and multiplying by one (value for ‘strongly disagree’). The
domain score was then standardised as a percentage using the following
formula:To measure inter-observer agreement across the ordinal categories
for each guideline and consensus statement, a weighted kappa (kw)
was calculated using SAS version 9.2 software. This takes into
account the degree of disagreement between the observers by
assigning less weight to agreement, as categories are further
apart.15,16 An overall kw was also calculated across all guidelines and
consensus statement. A kappa value of