There were several limitations to this study. No
territories responded to the survey in 2007. Two
territories responded in 2010. The inclusion of the
2 territories does not explain the decrease in PM or
QI since 2007. Measurement of the QI construct needs
continued refinement. There were several issues regarding
the measurement of QI in this study. First,
the definition of QI provided in the 2010 survey was
slightly revised from the 2007 definition. Second, construct
validity may begin to be assessed with the inclusion
of additional QI related questions. Specifically,
these additional QI related questions on the 2010 survey
facilitate an understanding that the survey question regarding the presence of an agency QI process may be
unclear ormisinterpreted by respondents. For example,
6 respondents indicated they had no QI process at their
agency but implemented more than 1 formal project
to improve quality, 7 respondents indicated having QI
in job descriptions but no QI process, and 5 respondents
provided staff training in QI but reported no QI
process. When one state was contacted to clarify their
response, the respondent commented that the question
itself inquires about the “agency” having its own QI
process. Their initial response was “no” despite having
QI partially implemented for specific programs. As
referenced in the “Methods” section, the response options
to this question included whether QI was fully or
partially implemented either department-wide or for
specific programs. Perhaps rewording the question to
ask the level ofQI implementation may provide greater
clarity. Finally, verification of actual improvements in
quality and performance management did not occur.
Validity and reliability of the constructs continue to be
limitations.