Limitations of the Literature
As with all systematic reviews, our results and conclusions depend on the quality of the published literature. Heterogeneity in outcomes, populations, study designs (or interventions), and measured outcomes was a problem for both Key Questions. This level of diversity in the knowledge base precluded us from pooling results statistically.
The limitations of the literature for Key Question 1 studies included:
Lack of a priori specification and inconsistent approaches to creating health literacy and numeracy levels or thresholds in analyses, hampering comparisons between studies.
Inconsistent choices of potential confounding variables in multivariate analyses.
Small sample sizes, making it impossible for us to determine whether null findings represented a true lack of effect or simply limitations in statistical power.
Studies in just one clinic or in other narrowly defined patient populations, rendering the applicability of findings to other settings or populations unknowable.
Use of health literacy tools that continue to focus primarily on reading ability.
The limited number of studies examining potential mediators of health literacy, such as self-efficacy, knowledge, or beliefs.
Few studies examining the role of health literacy on health disparities.
No studies examining differences in outcomes related to oral literacy skills.
The limitations of the literature for Key Question 2 studies included:
Lack of an adequate control or comparator group in many studies, limiting the ability to determine the true effect(s) of the intervention.
Measurement of multiple outcomes with insufficient attention to ensure that each had been adequately powered to detect a difference.
Testing interventions that combined various design features to mitigate the effect of low health literacy but offering no way to determine the effectiveness of individual components.
Failure to perform adequately controlled subgroup analyses that would elucidate differential effects of interventions in low- and high-health-literacy populations.
Failure to report adequately the intervention design features that would allow future content analyses of effective interventions.