QW2001 Paper 6T2 Mr. Keith B. Stobie (BEA Systems, Inc.) Automating Test Oracles and Decomposability

Key Points

• Patterns for verifying test case results: external vs. internal checking
• Patterns for how to check: external vs internal data (data driven testing)
• Test logging strategies

Presentation Abstract

All of the test patterns given below deal with test result handling.

The first patterns are complementary patterns for solving the same problem. They deal with internal (in-line check) versus external (batch check) checking of the results and internal (hard-coded) versus external (data driven) inputs and outputs. Batch check (or Benchmark) is particularly useful for changing the Judging pattern into Solved Example.

The second patterns are supplemental Test Automation Designs. Separable Tests (or Atomic Tests) deals with selection and independence of tests. All behavior (or whole function) deals with post-conditions of an Item Under Test (IUT).

All of these patterns also address test logging strategies.

About the Author

Keith directs and instructs QA and Test process and strategy. He plans, designs, and reviews software architecture and tests for BEA. His most recent project was BEA WebLogic Collaborate and previously WebLogic Exterprise. Keith was Test Architect at Informix designing tests for the Extended Parallel Server product and Manager of Quality and Process Improvement. With over 20 years in the field, Keith is a leader in testing methodology, tools technology, and quality process. He is a qualified instructor for Systematic Software Testing and software inspections. Keith is active in the software task group of ASQC, participant in IEEE 2003 and 2003.2 standards on test methods, published several articles and presented at many quality and testing conferences. Keith is an ASQ Certified Software Quality Engineer with a BS from Cornell University.