Per-Build Baseline Testing
Every continual build receives the following testing in addition to dev unit tests via Buildbots.
• Smoke tests (P0 automation)
• Performance tests
Per-LKG Day Testing
Each day, the following testing occurs against the last known good (LKG) continual build:
• Manual verification of a set of functional acceptance tests (this can be limited to one type of hardware per day).
• Functional regression automation is executed.
• Continual testing of top web app tests picks up the daily rolling build (both automated and manual).
• Stress, long-running, and stability tests are conducted on a rolling basis. Re-run the tests on daily builds until no issues turn up, and then move to weekly runs.
• Continual manual exploratory testing and tour testing.
• AutoUpdate test automation.