QW2001 Paper 2A1 Mr. James Tierney (Microsoft) Getting Started With Model-Based Testing

Key Points

• How to improve your effectiveness through model-based testing
• How to test with models from Day One of your project
• How to keep “growing” your models so that they keep finding bugs

Presentation Abstract

Traditional testing Traditional software testing consists of the tester studying the software system and then writing and executing individual test scenarios that exercise the system. These scenarios are individually crafted and then can be executed either manually or by some form of capture/playback test tool. But hands-on testing and handcrafted test scripts are labor-intensive and inefficient ways to test modern software. These methods of creating and running tests face at least two large challenges: First, these traditional tests will suffer badly from the “pesticide paradox” (Beizer, 1990) in which tests become less and less useful at catching bugs, because the bugs they were intended to catch have been caught and fixed. Second, handcrafted test scenarios are static and difficult to change, but the software under test is dynamically evolving as functions are added and changed. When new features change the appearance and behavior of the existing software, the tests must be modified to fit. If it is difficult to update the tests, it will be hard to justify the test maintenance costs. Model-based testing alleviates these challenges by generating tests from explicit descriptions of the application. It is easier, therefore, to generate and maintain useful, flexible tests. Modeling Modeling is a way to represent the behavior of a system. Models are simpler than the system they describe, and they help us understand and predict the system’s behavior. Models are a useful method of representing software behavior. Models provide an easy way to update tests to keep pace with applications that are constantly changing and evolving. In recent years, there has been a growing movement in software testing to use the information contained in explicit models of software behavior to make it simpler and cheaper to do testing. Model-Based Testing Model-based testing is a black-box technique that offers many advantages over traditional testing:

• Constructing the behavioral models can begin early in the development cycle.
• Modeling exposes ambiguities in the specification and design of the software.
• Simple models can be used for testing from the very first day of the project.
• Models embody behavioral information that can be re-used in future testing, even when the specifications change.
• The model is easier to update than a suite of individual tests.
• The models can evolve alongside the software and will continue to find bugs throughout the development cycle.

Getting Started With Model-Based Testing Using simple programmatic test tools and familiar applications, this presentation makes the case for intelligent, model-based test automation and shows how to apply it from the very first day of the product life cycle to deliver high-quality software.

About the Author

James Tierney moved into his current position of Test Architect for Microsoft Windows User Experience after being a Test Manager, Test Training Manager and Director of Test. He first got interested in Model-based testing in 1983 while testing at Fortune Systems in Silicon Valley. Finding Microsoft fertile ground for advanced testing ideas, he is helping create a company wide Model-based testing architecture. He has a couple of patents on Model-based testing, and has given presentations on MBT, Software Reliability Engineering, Poka Yoke (Mistake Proofing) Software, Testing, and Test Management.