Using acceptance criteria definition The acceptance criteria for requirements, used subsequently to produce detailed test plans and test data, should quantify the behaviour, the performance or some other demonstrable quality of the final end product. In order for implementation of an individual requirement to be considered acceptable, it is often necessary to meet more that one measurable criterion; this is why we use the plural, ‘criteria’, here. Acceptance criteria must be defined for all types of requirements (particularly functional and non-functional ones), and at the same level of granularity as the definition of the requirements themselves.
The development of acceptance criteria can clearly be linked to scenarios and storyboards (Techniques 50 and 51), and, in the case of functional requirements, use cases and use case descriptions (Technique 62). The use of scenarios particularly helps in defining the scope of a requirement along with its respective acceptance criteria. Defining the requirements and their related acceptance tests together can force earlier conversations with the users to take place, and ensure that more relevant detail is described than would be the case if one were not considering the acceptance criteria.
BUSINESS ANALYSIS TECHNIQUES
191
Even when it is not possible to allocate specific objective measures to a requirement, it is still important to try and allocate some scale of measurement that will enable the testing community to decide whether an acceptable standard has been achieved. For example, different individuals may have different interpretations of what ‘an intuitive system’ would look like, and it will be necessary to define a range of acceptance criteria for different groups of potential users. In some cases, where any potential criteria chosen seem very subjective, a more objective assessment may be obtained and verified by some recognised domain expert, independent accreditation body or standard piece of legislation such as the Disability Discrimination Act.
It is also necessary sometimes to extend the definition of the acceptance criteria for a specific requirement with some tolerance to allow for fluctuations in the working environment. For example, consider the following acceptance criteria for the handling of customer enquires.
The acceptable response time shall be no more than 3 seconds for 90 per cent of enquiries and no more than 5 seconds for the remainder.
The more detail provided and care taken where defining acceptance criteria, the more likely will be success in producing a quality implementation, and the less likely is confusion to occur during testing.
In addition to the clear definition of acceptance criteria for functional requirements, driven by discussions with business stakeholders and the use of scenarios, it is also important to define acceptance criteria for each individual non-functional requirement attached to the functional requirements, to ensure that the operation of the delivered solution achieves the required standard in terms of performance, availability, security, robustness and usability. In particular, definition of acceptance criteria in the tricky area of usability can be made more measurable by applying the acronym PLUME, which stands for measures of: