AOAC INTERNATIONAL
FINAL REPORT AND EXECUTIVE
SUMMARIES
FROM THE
AOAC INTERNATIONAL
Presidential Task Force
on Best Practices
in Microbiological Methodology
Contract Deliverable Due to
The U.S. Food and Drug Administration
Contract# 223-01-2464 Modification 12
August 10, 2006
481 North Frederick Avenue, Suite 500
Gaithersburg, MD 20877-2417 USA
Telephone: +1-301-924-7077
Fax: +1-301-924-7089
Internet e-mail: aoac@aoac.org
World Wide Web Site: http://www.aoac.org
AOAC INTERNATIONAL
AOAC INTERNATIONAL
Presidential Task Force on
Best Practices for Microbiological Methodology
US FDA Contract #223-01-2464, Modification #12
Task Force Final Report
August 10, 2006
TABLE OF CONTENTS
I. BPMM Task Force Final Report
II. APPENDICES
A. Detection Limits WG Report
B. Matrix Extension WG Report
• B.1 Matrix Extension Essential Organisms List
C. Sampling WG Executive Summary
D. Sampling WG Introduction
E. Sampling WG Enclosure A – Measurement Error
F. Sampling WG Enclosure B – Statistical Process Control
• F.1 Appendices for Statistical Process Control
G. Statistics WG Executive Summary
H. Statistics WG Report Part 1 – Developing Standards and Validating
Performance
I. Statistics WG Report Part 2 – Study Variables
J. Statistics WG Report Part 3 – Uncertainty
K. Statistics WG Report Part 4a – LOD50
L. Statistics WG Report Part 4b – LOD50 Spearmann-Kärber Worksheet
M. Classification Matrix
N. Recommendations for Future Research
O. Glossary
P. Task Force Membership
1. Steering Committee
2. Detection Limits Working Group
3. Matrix Extension Working Group
4. Sampling Working Group
5. Statistics Working Group
6. AOAC Staff
BPMM Task Force Final Report 8-7-06
Page 1 of 11
AOAC INTERNATIONAL
Presidential Task Force on
Best Practices for Microbiological Methodology
US FDA Contract #223-01-2464, Modification #12
Task Force Report
I. Background
During the past several years, issues have been raised about the limitations of the current
AOAC guidelines for validation of microbiological methods. These issues have included
the high rate of apparent false negative results when unpaired samples are used, the lack
of a definitive acceptable range for “fractional positive” results for qualitative studies and
the lack of appropriateness of the guidelines to bacterial toxins. A statistical task force
was formed in 2003 to try to address the statistical issues, especially in the case of
unpaired samples, and propose solutions. A set of recommendations was drafted, but as
yet the recommendations have not been adopted by the Official Methods Board of
AOAC. This task force did not address all the issues and concerns previously raised
relative to validation of microbiological methods, but focused on the issues of importance
to the US FDA as outlined in the objectives of the contract.
Modification #12 of USFDA Contract #223-01-2464 arose from discussions of the
limitations of the current AOAC microbiology guidelines and a proposal to re-evaluate
the AOAC guidelines was created. Modification #12 of the contract is focused on
developing recommendations on the best practices for validation of microbiological
methods by an international team of experts. The goal of the group was to consider the
technical and statistical aspects of the current AOAC guidelines and ISO 16140 and to
recommend new approaches where needed, without regard to harmonization, consensus
within AOAC INTERNATIONAL or consensus among international validating
organizations.
To lead the project, AOAC appointed Russ Flowers to Chair the Presidential Task Force
on Best Practices for Microbiological Methods (BPMM, hereafter referred to as Task
Force). A task force structure quickly took shape, comprising a Steering Committee (SC)
of key individuals with varying expertise and four Working Groups – Detection Limits
(DLWG), Matrix Extension (MEWG), Sampling (SAWG), and Statistics (STWG). The
working groups were chaired by Steering Committee members and populated by
international experts in that topic area. Great effort was expended to identify technical
experts from government, industry, reference laboratories, and academia with varied
backgrounds in food safety, quality assurance, clinical diagnostics, veterinary diagnostics
and engineering. Not surprisingly, some of these experts also serve on committees for
other standards organizations, such as ISO (International Standards Organization), ASTM
(American Society for Testing and Materials), CLSI (Clinical and Laboratory Standards
Institute, formerly NCCLS), and CEN, the European Committee for Normalization. Care
was taken, however, to select scientists and technical experts, without introducing
political agendas.
AOAC Contract Deliverable for
Contract # 223-01-2464 Modification 12
BPMM Task Force Final Report 8-7-06
Page 2 of 11
It is interesting to note that ISO TC 34 SC 9 is also considering a revision of the ISO
16140 guidelines and the recommendations from the BPMM task force will provide
valuable input to that process. The BPMM project is an important step in the
international harmonization of microbiological methods.
The objectives contained in Modification 12 of the US FDA contract were assigned to the
working groups as appropriate, and the task of the working groups was to address the
objectives by developing recommendations based on sound scientific and statistical
principles. The Steering Committee provided guidance to the working groups and served
as editors for the final reports. There were some topic areas that overlapped between
Working Groups and, therefore, for future publication purposes, the ideas and
recommendations of the task force would be best organized by topic area rather than by
contract objectives as contained in this report.
This report summarizes the recommendations of the task force and is supported by
appended working group reports, which provide the details behind the recommendations.
The goal of the BPMM task force, in the short period of time allotted for the contract,
was to determine the best practices for validation of microbiological methods and to
make recommendations for consideration and further research by AOAC and US FDA.
The Task Force did not attempt to create new microbiology validation guidelines as many
of the recommendations represent new approaches that must be further evaluated from
the perspective of practical application. There is no expectation of adoption of the
recommendations by AOAC INTERNATIONAL. After discussing the merits and
limitations of the BPMM recommendations, it is hoped that additional work will be
funded to further refine and practically evaluate the recommendations presented herein.
The Steering Committee recommends first that existing data be used to compare the
statistical recommendations to current practice, and then laboratory feasibility studies be
conducted to test proposed study designs and sample preparation techniques. These
additional efforts would be expected to lead to development of new detailed guidelines
for validation of microbiological methods that will be proposed for adoption by AOAC.
AOAC Contract Deliverable for
Contract # 223-01-2464 Modification 12
BPMM Task Force Final Report 8-7-06
Page 3 of 11
II. Executive summary
The Presidential Task Force for Best Practices in Microbiological Methods (BPMM)
makes the following recommendations relative to the objectives of Contract #223-01-
2464, Modification #12. A more complete explanation and justification for the
recommendations is given in the appended documents. A glossary of terms is found in
Appendix O.
Objective 1: Once a microbiological method has been validated for an array of specific
foods and specific strains of a microorganism:
a) To what extent can these results be extrapolated to other foods and other strains?
b) Are there abbreviated but scientifically/statistically appropriate
procedures/protocols by which a validation can be expanded to include additional
foods and/or strains?
c) How can methods be applied to specific foods, where no validation has been
performed?
The BPMM Task Force recommends new food sub-categorization schemes based on
proximate analysis, level and types of background microflora, presence of inhibitors and
other characteristics of food matrices that may affect microbial growth, recovery or
analytical procedures. Based on the new scheme, varying degrees of verification or
validation (from no verification or validation to harmonized collaborative validation)
would be required in order to apply a method to a new food matrix. The degree of
validation or verification is dependent on how closely related the new matrix is to
previously validated matrices and on the current validation status of the method (single
lab validated, multiple lab validated or harmonized collaboratively validated).
A list of essential reference organisms and toxins was compiled to address the issue of
variability of strains. The organisms and toxins represent antigenic and genomic
variability and are recommended to be used as part of the inclusivity testing as
appropriate for the method target. Other food-borne isolates should be added to the
inclusivity list based on the claimed application of the method.
Objective 2: What are the scientific/statistical bases for developing performance
standards against which the validation of methods should be based?
The Task Force recommends that performance standards be based on public health
objectives (PHOs) and/or fitness for purpose criteria. In general, statistical methods
should be used to assist in setting realistic performance standards. These procedures
should be based on control of error related to a true negative testing positive (Type I) and
error related to a true positive testing negative (Type II). Levels of poor performance that
must be detected (with stated probabilities) should be determined. Appropriately
determined sample sizes should be used to meet the stated goals. This approach would be
a change from current practices where studi