11th INTERNATIONAL
SOFTWARE QUALITY WEEK
(QW'98)
26-29 May 1998, San Francisco, California USA

PAPER AND PRESENTATION ABSTRACTS

The 11th International Software Quality Week Technical Program is organized into separate tracks depending on the content of the presentations. Below are abstracts for all QW'98 half-day Tutorials, Keynotes, QuickStart Mini-Tutorials, and Regular Technical Papers. Updated 13 April 1998.

Mr. Larry Apfelbaum & Mr. John Schroeder(Teradyne Software & Systems Test)
Paper Title: Reducing the Time to Thoroughly Test a GUI (7A1)

Most modern applications include a Graphical User Interface (GUI), these interfaces can be extremely simple to agonizingly complex depending on the application. In many companies the testing of the GUI interface is a critical part of the plan for product release. The use of a graphical model to describe the behavior of a GUI combined with an automated test generation system can dramatically reduce the time required to test an application. This paper will describe a straightforward process used to build the model for a GUI and then generate suites of tests to verify the application's functionality. The results of an automated approach can be seen in the reduced time required as well as an increase in the testing thoroughness.
JUMP TO TOP OF PAGE
RETURN TO QW'98 PROGRAM

Mr. Bill Asburry (Dynamic Software Technologies, Inc.) [USA]
Paper Title: Integrating Rapid Application Testing with Rapid Application Development (3S2)

Rapid Application Testing addresses the real time testing of software as it is developed.

Minimums of (4) attributes are required to support an average size development project:

  • Knowledgebase of Best Practices
  • Script Automation to interrogate code, correct, and track
  • True Ease of Use
  • Shareable Project Guidelines
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Dr. David A. Banks, Dr. Paul Black, Dr. Leonard Gallagher, Dr. Charles Hagwood, Dr. Raghu Kacker, Ms. Lynne Rosenthal & Mr. James Yen (NIST) [USA]
    Paper Title: Software Testing by Statistical Methods (8T2)

    Software testing is hard, expensive, and uncertain. Many protocols have been suggested, especially in the area of conformance verification. In order to compare the efficacy of these protocols, we have implemented a designed simulation experiment that examines performance in terms of testing costs and risks. We find that no single method dominates all others, and provide guidance on how to choose the best protocol for a given application.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Boris Beizer (Independent Consultant) [USA]
    Tutorial Title: An Overview of Testing -- Unit, Integration, System -- Outline and Index (TE)

    This is an overview of the testing field. Its purpose is to provide you with the technical and conceptual vocabulary of testing. Testing has emerged as a field within software engineering and has acquired a big vocabulary. It has progressed, in the past 20 years, from intuition to science -- from personal heuristics to well-understood practices rooted in theory and confirmed by use and experiments.

    Paper Title: Prioritizing Your Y2K Testing Effort: Debunking the Special Dates Myth (5P1)

    Software testing under ordinary circumstances is acknowledged to consume half of the software development labor content. Because testing is one of the principal means of discovering Y2K sensitivities and because of the massive amount of regression and future-time testing needed, total testing labor content for Y2K remediation projects could easily be in the 75% - 80% range. There will not be enough time and enough human resources to do it all -- prioritization is essential. One of the more popular testing approaches taken concentrates effort on special dates such as Jan. 1, 2,000 or Sept. 9, 1999. Such special date testing is largely fueled by Y2K myths rather than based on substantive evidence. Noted testing expert Beizer will provide guidelines on prioritizing your Y2K testing effort and how, if any, special date testing should be taken into account.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Larry Bernstein (National Software Council) [USA]
    Paper Title: Trustworthy Software (3Q)

    Software is hard because it has a weak theoretical foundation. Most of the theory that does exist focuses on the static behavior of the software -analysis of the source listing. There is little theory on its dynamic behavior - how it performs under load. To be trusted software must work reliably and predictably under load.

    Even after we find and fix a bug, how do we restore the software to a known state, one where we have tested its operation? For most systems, this is impossible except with lots of custom design that is itself error-prone.

    The four measures I use to tell just how good my software systems perform are reliability, success with unexpected stimuli, true system capacity and the number of service calls. They all measure the system from the viewpoint of its field execution. The study of the dynamic behavior of software addresses these issues.

    Most of the theory for software technology today focuses producing the source code and with the exception of testing there is very little work on how it performs in the field. This tutorial treats the disciplines of software dynamics and how they relate to making software trustworthy. The topics of software fault tolerance, software rejuvenation, performance modeling, robust testing and stability will be covered.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. James R. Bindas (Intel Corporation) [USA]
    Paper Title: Lesson Learned: Transferring Software Development Best Known Methods (BKMs) Between Product Lines (STAND-BY)

    Transferring Best Known Methods (BKMs) between product lines is a sound idea. However, implementing the transfer is a non-trivial task. Challenges surface that include: varying definitions of BKMs, leveling the expectations and definitions of the individual BKM, packaging BKMs, certifying BKMs to a certain criteria, and placing BKMs in a central location.

    These unique challenges require unique solutions. This paper addresses these solutions by:

  • Explaining the creation of an infrastructure to support BKM transference.
  • Addressing the human aspects between BKM suppliers and customers.
  • Certifying that BKMs are transferable because they include a total package of methodology, tools, and lessons learned.

    The approach detailed may not be optimal, but it mirrors real life by highlighting successes and failures. The approach is offered as a building block for individuals involved in this arena, so that they will not incur mistakes already uncovered.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

  • Mr. Robert V. Binder (RBSC Corporation) [USA}
    Tutorial Title: Testing Strategies for Object-Oriented Systems (TI)

    Participants in this half-day tutorial will learn about UML-based test design for object-oriented systems at the class, cluster, and subsystem level.

    Classes (objects) have distinctly different behavioral patterns (modes). A mode characterizes domain and behavior and must be identified to select an effective test strategy. This tutorial presents new approaches for domain/state modeling and producing effective test suites from these models.


    Paper Title: Testing Object-Oriented Systems: Best Practices (10P2)

    Presents highlights of current best practices for testing object-oriented software. Six key factors for effective Object-Oriented Testing are defined. The state of the art and challenges in each is considered. Suggestions for process improvement are developed from this analysis.
    (Click here for Bob Binders's Home Page)
    (rbinder@rbsc.com)
    (TOP OF PAGE)
    RETURN TO QW'98 PROGRAM

    Mr. Larry Boldt (Technology Builders, Inc.) [USA]
    Paper Title: Y2K Requirements-Driven Automated Testing...Do You Have a License to Drive? (2S2)

    This presentation discusses, in detail, how requirements management became the focus of Y2K certification of systems. It uses a real life business project as the basis for describing a "how-to" approach for defining requirements, generating test plans/cases, and creating automated test cases from the requirements. The presentation shows the blending of business process reengineering (BPR) techniques and automated software quality (ASQ) techniques to solve a problem that plagues a majority of Y2K software development projects.

    Attendees will gain an understanding of:

  • Using business requirements to drive Y2K testing
  • Using an automated repository for managing requirements
  • Developing a requirements-to-test matrix showing what to test
  • Developing a test plan driven by requirements
  • Using an automated testing tool to create regression tests that can be used beyond Y2K
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Mr. Michael Bowden (CYRANO) []
    Paper Title: Year 2000: A Practical Approach to Reducing Business Risk (2S1)

    (Abstract to be supplied)
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. David W. Carman, Mr. Siddhartha R. Dalal, Mr. Ashish Jain & Mr. Nachimuthu Karunanithi (Bellcore) [USA]
    Paper Title: A Test Generation Factory for Year 2000 Testing (9A1)

    We present a new Factory based process developed at Bellcore for the year 2000 testing of renovated systems. We leverage the key principle that essential business logic does not change during year 2000 renovation and, hence, reinvesting in creation of brand new testcases for the year 2000 compliance testing of a system is expensive and redundant. Using the proposed process, a set of user defined rules, and a set of in-built system rules, a model for the date of dependent business logic is extracted from the existing set of regression test cases for the system. In absence of regression testcases, the process works equally well with test data files and/or usage scenarios captured using a capture/replay tool. The extracted model is then populated with year 2000 sensitive dates with the proviso that the business rules of the application are not violated. New testcases are then generated using one of several generation strategies proposed.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Eugenio Cervetto (Performance Research) []
    Paper Title: Performance Evaluation of Real-Time Embedded Software Applications using PREDICTA (7S1)

    In this presentation the description of a recent experience in applying a performance prediction and control methodology to the development of critical real-time embedded systems is given. A modeling and prediction tool named PREDICTA and developed by Performance Research supports the described methodology.

    First a short description of the embedded systems scenario from the performance problem perspective is given together with an overview of the related risks and problems. Some of the most outstanding limitations in the present development process adopted by most real-time systems developers are outlined.

    Starting from Performance Research's experience in this field a short analysis of the needs for improvement raised by some large European customers is presented. An overview of the proposed approach that, in a variety of cases, has been proven to be effective in enabling our customers to successfully cope with these well-known "performability" risks is also provided.

    Moreover, a short functional description of the Predicta tools is given as well as an overview of the goals, advantages and measurable results coming from the application of the tool in the development of real-time systems.

    Some words are then spent in providing a characterization of the performance defects, of their origin in the development process and of the related risks and criticalities, while some key ideas on how a correct application of the proposed methodology can overcome these specific problems is given.

    Finally, some background information on the use and support of the Predicta tool in different application areas is provided.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Antonio Cicu, Mr. Domenico Tappero Merlo, Mr. Francesco Bonelli, Mr. Sandro Francesconi, Mr. Fabrizio Conicella & Mr. Fabio Valle (MetriQs Srl / OSRA SISTEMI) [ITALY]
    Paper Title: Managing a Customer's Requirements in a SME: A Process Improvement Initiative Using an IT-based Methodology and Tool (8S2)

    The paper describes:

  • the context (a small software house) of a process improvement initiative aimed to improve the management of customer's requirements
  • the adopted guidelines, derived from: ISO/IEC 12207 and ISO/IEC PDTR 15504 (SPICE) (process model), IEEE J-STD-016-1995 (documentation), ami and GQM (improvement measurements)
  • the measurements results
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Mr. James M. Clarke (Lucent Technologies) [USA]
    Paper Title: Automated Test Generation from a Behavioral Model (2T1)

    The challenge for testers: reduce the testing interval without reducing quality. One answer: find a new way to approach test design and test generation. This paper will discuss an ongoing Lucent Technologies experiment in automated test generation from a behavioral model of the software product under test. Results indicate that our new approach can increase the effectiveness of our testing while reducing the cost of test design and generation.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Ms. Maxine Crowther & Mr. Dave Oliver (Cadence Design Systems, Inc.) [USA]
    Paper Title: Automating Defect Tracking and Reporting: A Solution for the 21st Century (2A2)

    This paper describes:

  • How to choose the right business model for defect tracking
  • How to automated the model
  • How to report metrics and operational reports across a diverse Software Engineering population
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Ms. Hazel Curtis, Ms. Diana Burkhardt & Mr. Alfred Vella (Allstor Software / University of Luton) []
    Paper Title: Automated Test Suites from Reverse Engineering and Planguage (8A2)

    This paper describes a process which derives a formalised natural language specification from code. This specification is then inspected manually and amendments made using expert knowledge. Finally, the corrected specification is used to automatically generate test cases. The process described avoids the problem of generating self-fulfilling test cases from code.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Gregory T. Daich (SAIC/STSC) [USA]
    Paper Title: Essential Year 2000 Practices (2M2)

  • Focus on essential testing practices helps minimize rework and complete our Year 2000 computer system upgrades in time.
  • Corporate-level and project-level guidebooks and planning templates enables organizations survive the year 2000 challenge.
  • Designing, documenting, and executing comprehensive, project-specific, date-related tests exercises due diligence (and will help us sleep well on Jan. 1, 2000).
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Mr. Manu Das & Mr. Drew Slotnick (SOFFRONT Software, Inc.) [USA]
    Paper Title: The Revolution in Defect Tracking and Analysis, Software Quality and the Internet (6S2)

    Current defect tracking products address corporate needs by including features like automatic e-mail notification of key personnel of changes to defect records, customizable databases, synchronization of multiple databases, integration with version control programs, advanced reporting and analysis capabilities, and attachment of documents and test files to defect records.

    The need for greater communication among widely dispersed development groups on complex projects and the advent of the Internet and intranets led to the emergence of the Web-enabled defect tracking systems. These systems allow users contact with the defect information and the ability of downloading maintenance updates.

    We'll look at these points in detail:


  • Evolution of tracking tools and the forces driving change
  • Current Feature sets for common defect tracking and analysis products
  • Internet technologies: accelerating the rate of change
  • Tracking systems integrating web technologies
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Mr. Michael Deck (Cleanroom Software Engineering, Inc.) [USA]
    Tutorial Title: How Testers Can Use Formal Methods to Analyze and Improve Software Requirements Specifications (TD)

    This tutorial will show testers and other non-developers how they can use formal methods to analyze and improve software documents that they receive from others. One of the significant problems presented to testers is having to work from inadequate, incomplete, ambiguous specification documents. This tutorial will introduce informal ways that testers can "think formally" about specification documents to reveal domain gaps, incompleteness, and other problem spots before they have too great an impact on the testing process. The method that will be used is the functional/denotational approach pioneered by Harlan Mills of IBM. We will supplement it with a model-based approach to data abstraction.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Ms. Claudia Dencker (Software SETT Corporation) [USA]
    Paper Title: A Tester's Perspective of a Y2K Project at Hewlett-Packard (6M2)

    As the clock marks time to the year 2000, corporations throughout the world realize key corporate functions are in jeopardy. Will their employees be paid correctly given a new millennium and tax year? Will their nightly processes execute after midnight and in the early hours of January 1, 2000? How will related dependent systems fair? What are the back-up plans?

    Here is a project as told from a tester's perspective. The project is a payroll system of a major US corporation, Hewlett-Packard. This paper covers the issues important to Test, such as how the project was staged, phases of test, the role of test automation, and other tools. With every project there are challenges and these will also be shared as will some interim lessons learned.

    There is still time to realign testing strategies and to get the job done as time marches unrelentingly to midnight, December 31, 1999.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Thomas A. Drake (Booz Allen & Hamilton) [USA]
    Paper Title: Is Quality Really "Good Enough" for the Millennium? (2Q)

    Engineering excellence in software development and for information technology at large continues to be a largely elusive goal for many in the industry. With the challenges posed by the Year 2000 "bug", the continuous upgrade treadmills still characterized by defects and flaws, the siren call of "silver bullet" solutions, and the market's increasingly short cycle times, where does quality fit in anymore, if at all?

    Many organizations have made great strides in improving quality and a number of world class development houses have emerged over the last few years, so what makes quality quality? Is quality merely perception and subjective opinion? What is the basis for stating what quality is in the first place?

    Quality IS the middle name in software quality engineering and it takes a combination of people, process, and technology to make it happen. And software has become a critical national resource upon which much of a nation's security, prosperity, and economic competitiveness increasingly rests.

    This mini-tutorial will introduce the business case for software quality engineering, and the absolutely crucial role of the people and the process combined with an automated practice-oriented, process-driven, product-level measurement and analysis activity as THE critical foundation for delivering robust, stable, and functionally fit for use, software based information technology systems for the 21st century.

    The intent of the tutorial is to provoke a lively and interactive conversation about the difference quality can or should make in our industry.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Dave Duchesneau (Boeing Commercial Airplane Group) [USA]
    Paper Title: Design for Test (Or, How to Make it Hard for the Pernicious Bugs to Hide) (10P1)

    Given the increasingly mission-critical "bet-your-business" nature of software systems, it's only natural to expect software to become increasingly reliable. Yet the opposite is true, often as a side-effect of dramatic increases in system complexity. As often as not, the lack of enough time for testing is cited as reason that bugs occur with such frequency in the field (after all, the users seem to have no problems finding them). However, even with ample time to catch the most "important" bugs, they still surface with amazing spontaneity. This session will show that, in reality, system developers provide ample opportunities for bugs to hide. If we are to improve our bug (defect) detection ability during testing, then testers really ought to know where they hide. Furthermore, if we want to encourage defect prevention, then developers ought to know how to avoid creating such nifty hideouts.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Dietmar Ernst, Mr. Frank Houdek & Mr. Thilo Schwinn (University of Ulm / Daimler-Benz AG) []
    Paper Title: An Experimental Comparison of Static and Dynamic Defect Detection Techniques (STAND-BY)

    In our experiment, we compared test and review for the field of embedded systems. Important results were that reviews led to the discovery of more errors than a combination of black-box and white-box test. When taking the expended efforts into consideration, the static technique performed substantially better than the dynamic one.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Fabbrizio Fabbrini, Mr. Mario Fusani, Mr. Vincenzo Gervasi, Ms. Stefania Gnezi & Mr. Salvatore Ruggieri (IEI-CNR / Dip. di Informatica) [ITALY]
    Paper Title: Achieving Quality in Natural Language Requirements (4A1)

    We present a proposal of a quality model for natural language requirements that concentrates on linguistic properties of requirements documents. We deploy the quality framework of Krogstie et al. as a superordinate rationale supporting the design of a quality model. At the same time, the quality model is intended to cover and classify the corpus of linguistic techniques proposed in the literature for the analysis of requirements. As a result, we have identified a number of factors and criteria that affect the overall linguistic quality of requirements documents. Our model can thus be used to choose the most appropriate techniques to apply in order to reach a desired quality level.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Tom Gilb (Independent Consultant) [NORWAY]
    Tutorial Title: Evolutionary Delivery Project Management (TH)
    Paper Title: The Alternatives and Supplements to Conventional Testing (4Q)

    The Waterfall Model has been the most commonly taught and assumed project management model. But in spite of good intentions, it may be the cause of lack of project control. New Department of Defense and IEEE standards reject the Waterfall Model and assume the Evolutionary project management model. Major companies like HP and Microsoft teach and use the evolutionary project management model, and years of experience indicate it is far more successful in project management. The evolutionary model (Evo) is characterized by many frequent partial result delivery steps to some form of user or customer. The resulting feedback is used to improve requirements, design and processes. This tutorial will give the basics of evolutionary delivery.
    JUMP TO TOP OF PAGE
    RETURN TO QW'97 PROGRAM

    Mr. Robert L. Glass (Computing Trends) [USA]
    Paper Title: The Software Crisis: Is it for Real? (1P1)

    Much has been written about the "software crisis" - that software is always behind schedule, over budget, and unreliable. Claims of such a crisis are often sued to support both vendor and researcher claims of "solutions". But there is something wrong with these claims:

  • This is the "computing age". Our society functions successfully on a platform of computers fueled by software. How could this be true if most software projects fail?
  • There is very little data to support the cries of crisis, and the data that does exist varies enormously.
  • Most schedule and budget problems in the software field are caused more by our inability to estimate than by poor performance on software projects.

    This presentation is contrarian, and the speaker welcomes audience interaction.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

  • Mr. Steve Goldstein (Genetics Computer Group) [USA]
    Paper Title: Strategies for Testing a Web-based Application: Divide, Conquer & Automate (6A1)

    Our company has just released a new product: a web interface to our software. We found no off-the-shelf testing tool that would address all our validation problems, so we crafted our own tools.

    In this talk, we present the design of our testing scheme and our analysis of web applications that led to such a design. Although some parts of our testing solution are specific to our product, the analysis and its implications are widely applicable.

    The key point in the analysis is that although a web application may be a complicated system, less complicated subunits can be isolated and tested independently. Much of this testing can be automated.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Manuel Gonzalez (Hewlett-Packard) [USA]
    Paper Title: Process Improvements Via Testing Results: A Case Study (3M2)

    In this paper we present the implementation of a plan to improve software processes using results gathered during the testing phase in HP Large Format Printers projects. Most of these results are just used to identify and fix defects but we think that data about defects and other quality-related data (that can be collected without too much effort) are very useful to draw conclusions about process faults.


    Paper Title: System Test Server through the WEB (6A2)

    In order to reduce cost and improve reuse rate we propose a testing automation environment integrating several techniques:

  • To create a knowledge database as the core of the testing automation environment which groups the test cases and enough information to execute them.
  • To use Web technology in a manner which reduces client setup to a minimum and to deploy the maximum testing through all the company. o To share testing resources and simulation software, so to allow to clients access to expensive resources (due to development time or hardware/device availability).
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Ms. Martha Gray, Dr. Kathleen A. Kegley & Ms. Lynne Rosenthal (NIST) [USA]
    Paper Title: Applications of Formal Specification Languages in Conformance Testing (8A1)

    The potential role of formal specification languages in conformance testing is being evaluated by the Software Diagnostics and Conformance Testing Division at the National Institute of Standards and Technology (NIST). Open issues, experience reports, and an evaluation of state-of-the-art testing systems based on formal specification languages are presented.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Matthias Grochtmann Mr. Joachim Wegener & Mr. Roman Pitschinetz (Daimler-Benz AG) []
    Paper Title: Integrated Test Management within the Tool Environment Tessy (6T2)

    A software testing tool is most effective when it works within an organizational context. Computer-aided test activities framed into a testing life cycle management reduce time, effort and cost. The test system TESSY provides support for both automation of dynamic unit testing and appropriate management of the whole testing life cycle. Practical trials of the test system started in 1994. First promising results indicate that the test efficiency can be enhanced significantly by using TESSY. Future work will focus on extensions of TESSY with respect to integration testing and automatic generation of test cases and test data.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Avi Harel (ErgoLight Ltd.) []
    Paper Title: Automation in Usability Validation (6S1)

    In this article the term "usability" refers to friendliness, ease of use, ease of learning, user productivity and the system resistance to user errors. Usability validation should rely on the examination of real end users when they do their real work, rather than on GUI specification, standards or style guides. Available tools for Software Quality Assurance (SQA) do not address the issue of resolving problems that end users encounter during operation.

    Automation in usability testing is required to satisfy three main needs:

  • To reduce the costs of testing procedures;
  • To shorten testing cycles;
  • To identify severe usability problems that are difficult to detect manually.

    Software tools used in usability testing should be integrated with human factors methodologies and should support full cycle application development.

    ErgoLight integrates methodologies of Human Factors Engineering in software tools. ErgoLight allows QA experts to identify problems in the user interface design, in the user education and training programs, in the user documentation and in the on-line help system. The problems ErgoLight identifies are of user conceptual difficulties, using the wrong terminology, orientation problems caused by mode discrepancy and wrong response to user errors. For all problems identified, ErgoLight provides backtrack to the record of user actions. For repeated difficulties, ErgoLight provides statistics of the user wasted time, used as a measure of costs of usability deficiencies.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

  • Mr. Hans-Ludwig Hausen (GMD Gesellschaft fur Mathematik und Datenverarbeitung mbH) [GERMANY]
    Tutorial Title: Software Metrics for Procedures, Objects and Agents (TF)

    This presentation will review the most relevant quality modelling approaches that are applied today, and will demonstrate how quality issues for the web can be integrated into today's industrial practice.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Michael Heffler & Mr. Robert Thien (Bellcore) [USA]
    Paper Title: Providing a Context for Process Improvement and Assessment (7M1)

    Software process improvement (SPI) efforts either don't get started or fail because the effort is not strongly tied to financial results. Since most companies don't keep metrics on how they are doing, it's difficult to show how SPI is helping, particularly when results occur over time. This talk will discuss ways to tie results to SPI, get quick wins, and establish a successful SPI effort.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Douglas Hoffman
    Paper Title: A Taxonomy for Test Oracles (8Q)

    Automation fo testing is often a difficult and complex process. The most familiar aspects of test automation are organizing and running of test cases and capturing and verifying tet results. Generating the expected results is often done using a mechanism called a test oracle. This paper describes several classes of oracles created to provide various types of verification and validation. Several relevant characteristics of oracles are described and the advantages and disadvantages for each type of oracle are described.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Prof. William E. Howden (UCSD) [USA]
    Paper Title: Software Confidence, Computational Integrity, and Statistical Model Checking (8T1)

    This paper reviews different methods for model checking, and the kinds of models and integrity properties to which they have been applied. It indicates the relationship between currently popular model checking ideas and older ideas from static analysis. The use of model checking methods in computational integrity analysis is described, including systematic techniques for incorporating the identification of integrity properties in a software development process. The paper then reviews the basic results in statistical testing. New results are described, including: the role of historical precedent in confidence estimation, the use of failure concentration in accelerating the computation of confidence, and the development of more optimistic formulae for programs that are minimally reliable. Finally, the paper goes into detail on how these two classes of methods can be combined to produce an approach to computational integrity that includes both complete and partial measures of confidence. Examples are included of its application both to abstract system models and to actual programs.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Jon Huber (Hewlett-Packard) [USA]
    Paper Title: Developing Metrics for a Software Testing Organization (9M1)

    A basic principle of archeology is that the artifacts found in a culture represent the values of that culture. This leads to a question. What are the "values" of Software Testing Organizations and are they what they should be? The values of the Boise Hewlett Packard LaserJet Test Lab were explored in a Goal-Question-Metric workshop held February 12, 1998. The paper and presentation will contain results, observations, recommendations, and conclusions from the workshop.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Ms. Alka Jarvis () []
    Paper Title: Applying Software Inspections - Real-Life Experience at Cisco Systems (9S1)

    Inspections in the software industry have been a proven technique to improve the quality of the product and also to increase the productivity. The purpose of inspections is to identify and fix defects in a deliverable of a software development life-cycle. The deliverable can be a requirements document, design document, code or test plans. The idea is to detect the defects early in the development cycle to control the cost and schedule. Even though conducting inspections is not a new practice, few organizations can boast of conducting regular inspections with a precise process. Most companies shy away from adopting this useful methodology either because of culture or failure to know how to implement the process.

    The Wide Area Network Business Unite (WANBU) of Cisco Systems has been consistently conducting inspections for past three years. The presentation would be a case-study of Cisco-WANBU, highlighting the successes and difficulties of Cisco-WANBU in implementing the process of inspection. Presentation will cover how Cisco-WANBU accomplished the following:

    The presentation is intended for individuals who are tired of "theories" and are looking for practical "How To" approach on implementing a new process based on real-life experiences of another organization. It is especially for management staff of software research & development departments, software development engineers, software testing and quality assurance personnel, project managers, quality systems and ISO 9000-3 staff and quality control engineers.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Ieuan E. Jones (Royal Military College of Science Shrivenham) []
    Paper Title: Application of Quality Practices to Web-Site Development (4S1)

    To a company at the turn of the 20th/21st Century, the World Wide Web (the web) is becoming, or indeed has become, as fundamental to their business as the telephone or telegraph became to companies at the turn of the 19th/20th Century. This paper highlights three issues that may affect the quality of a website: the use of tools to facilitate development; managing the change and evolution of the site as it is developed and the design process used in development. Regarding tools, the paper identifies some of the problems encountered and reports upon early experiences of teaching HTML from basic principles. Similarly, the need to manage the evolution of change, in a medium where change is so easy, is discussed. A mechanism for carrying configuration management information is proposed, and early results from some simple experiments presented, along with a proposal for further investigation. Finally, the paper questions the design processes employed, and questions whether the design of web sites can be managed solely as a software project or whether another model is needed to accommodate the more graphical and artistic aspects of development.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Oliver Jones (Rational Software Corporation) [USA]
    Paper Title: System Testing for Java-based Internet Applications: How, When and Why? (3S1)

    Developers using Java to build transactional Internet applications are faced with daunting quality assurance challenges. These web sites are often intended for use by enormous numbers of people, using many client and server configurations, and a variety of arrangements of clients, web servers, and database servers. The inherent complexities of these applications and the exposed risk of an application breaking in a customer's hands mandates the need for testing early and often in the design, development and deployment phases.

    This session discusses the testing stages needed to get ready for high-volume rollout of Java-based information systems:

  • Structural testing for the Web sites underlying the Java-based applets
  • Functional and regression testing for Java applets at the unit-testing level
  • Functional and regression testing at the system (server/browser/applet) level
  • Performance testing for the Web servers
  • Performance testing for second-tier application and database servers
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Dr. Cem Kaner (Attorney at Law) [USA]
    Paper Title: Year 2000, How Can I Sue Thee? Oh, Let Me Count the Ways (1P2)

    It's getting common to hear that the legal-expense cost of Y2K will be around $1 trillion, even more than the $600 billion that the pundits tell us we will see spent on the technology costs. How real is this?

    I'm not a Y2K expert, just a lawyer with a lot of software experience. But I've been reading Y2K lawyers' articles, and listening to self-appointed Y2K litigation experts with a growing sense of disquiet. I think that some people (some lawyers and many computing consultants) are pushing and stretching legal issues as far as they can go in order to milk extra money from terrified clients. I also think that all this jumping around screaming "lawsuits here, lawsuits here" is bound to attract some sharks to our waters. Oh the joys of the self-fulfilling prophesies.

    In the midst of all this baloney and sharkbait, there will also be many well justified disputes over serious problems that will need fair legal resolution. Some of those disputes will yield lawsuits, and some few of those lawsuits will actually be filed against deserving targets.

    So, here I am, a lawyer who is definitely not a Y2K expert, trying to sift through the hype, trying to figure out how to make the world a safer place for the clients that I'll represent in Y2K. This talk is my progress report--I've made sense of some issues, found some strategies and safe harbors and good sources of information. It's not even close to a thorough understanding, but it's more than I knew a year ago, and more than many of my clients know today.


    Paper Title: Concise Test Planning (9Q)

    (Abstract to be supplied)
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Zachi Karni, Mr. Dror Orel & Mr. Shmuel Ur (IBM, Haifa Research Lab) [ISRAEL]
    Paper Title: Using 3D to Visualize Dynamic Path Coverage (4T2)

    In this paper we illustrate an innovative 3D visualization technique of coverage information and explain its advantages over 2D visualization. We explain how 3D visualization of statement and branch coverage can improve testing. We show, using the forward path coverage criteria, how graphical information can capture dynamic properties of program execution. 
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Richard Kasperowski (Altisimo Computing) [USA]
    Paper Title: Automated Testing and Java Class Libraries (9T1)

    Commercial software companies develop and market large, complex software systems. Major releases and maintenance updates are distributed at high frequency, as often as once per month. With neither enough time nor enough people to test the software before shipping it to customers, how can you deliver a high quality product? The answer is to use a testing strategy to acheive 100% feature coverage and to automate as much testing as possible.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Ms. Karen S. King (Sequent Computer Systems) [USA]
    Paper Title: Ensuring Quality in Software Suppliers (4M2)

    As we approach the year 2000, software complexity is increasing exponentially, and software companies are more frequently leveraging code developed by other companies. This paper will discuss methods of ensuring that the quality of software provided by other companies is predictable
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Ed Kit (Software Development Technologies) [USA]
    Tutorial Title: Automating Software Testing and Reviews (TG)

    In the rush to automate software testing, many people forget to pay attention to the basics -- i.e., ensuring they have a valid and effective software test process and prioritizing their automation needs for greater chance of success. Edward Kit addresses these issues and concerns when he presents:

  • Integrating Tools and Your Testing Process
  • Fixing Three Common Pitfalls of Capture/Playback
  • The Potential Synergy of Five Key Testing Tools
  • The Importance of Technical Reviews
  • Key Software Testing Success Factors
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Mr. Martin Klaus (TakeFive Software) []
    Paper Title: Beyond Testing: How to Fix Year 2000 Critical Code (8S1)

    Testing is the essential approach of determining if an application is ready for the Year 2000.

    However, without reading and understanding the entire code base--possibly a million or more lines of code--how do you find out what parts of your program cannot handle "00" as the year 2000?

    This paper goes beyond testing. It explains the Y2K-critical parts of C and C++ applications, gives developers practical advice about identifying and fixing Y2K-problematic code, and outlines how to identify the right tools for finding, managing and fixing Y2K bugs.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Peter Liggesmeyer & Mr. Martin Rothfelder (Siemens AG) [GERMANY]
    Paper Title: Going Beyond Correctness: Improving Software Robustness (3A1)

    (Abstract to be supplied)
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Leslie Allen Little (Aztek Engineering) [USA]
    Paper Title: Why Johnny Can't Read or Write Requirements (STAND-BY)

    Why can't Johnny read or write requirements? In 1955, Rudolf Flesch wrote Why Johnny Can't Read which delivered a stingy indictment of using the wrong method to teach the principles of reading.

    Now, some forty plus years later, the same can be said of reading and writing requirements and again the underlying principle is that we aren't teaching it correctly. More clearly stated, Johnny can't read or write requirements because it isn't clear to him

    1. what needs to be written
    2. how the requirements he writes will be read
    3. why the requirements he does write are always out-dated

    This paper investigates these questions, proposing the use of a tool, ReqTrack, along with the adoption of a process, ReqProc, to overcome Johnny's deficiencies. The issues are examined in the context of the application of ReqTrack and ReqProc on a real time embedded software project. Examples are provided in each case. It is the authors belief and limited experience that the application of these processes and techniques will allow Johnny to start writing and reading more useful requirements.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Michael R. Lyu (The Chinese University of Hong Kong) []
    Tutorial Title: Current Techniques and Tools for Software Reliability Engineering (TB)

    This presentation:

  • Defines the concepts and techniques behind software reliability measurement.
  • Surveys and evaluates software reliability models.
  • Describes current software reliability tools and demonstrates their usage and application.

  • Paper Title: A Phase-Based Approach to Integrate Reliable Software (STAND-BY)

    In this paper we will address the development, testing and evaluation schemes for software reliability, and the integration of these schemes into a unified and consistent paradigm. Specifically, techniques and tools for the three phases of software reliability engineering will be described -- the three phases are (1) modeling and analysis, (2) design and implementation, and (3) testing and measurement.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Brian Marick (Testing Foundations) [USA]
    Paper Title: When Should a Test Be Automated (6Q)

    This presentation is intended to help the individual tester decide whether a particular test should be automated or manual. I will describe how automated tests find bugs (it may not be the way you expect), how product structure affects the value of automated testing, and what questions you should ask before automating.
    marick@testing.com
    http://www.stlabs.com/marick/root.htm
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Prof. W. Douglas Maurer (George Washington University) [USA]
    Paper Title: Program Correctness and the Year 2000 Problem (2M1)

    There is a continuing misunderstanding on the part of most people in the computer field as to what program correctness is about. Our hope, expressed in this paper, is that the Year 2000 problem will provide a more easily understood explanation of what correctness is, and what it is not.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Edward Miller (Software Research, Inc.) [USA]
    Paper Title: The WebSite Quality Challenge (4S2)

    Because of its possible instant worldwide audience a WebSite's quality and reliability are crucial. The very special nature of the WWW and WebSites pose unique software testing challenges. Webmasters, WWW applications developers, and WebSite quality assurance manages need tools and methods that can match up to the new needs. Mechanized testing via special purpose WWW testing software offers the potential to meet these challenges.
    (Complete Paper)
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Stefan Mohacsi, Mr. Armin Beer & Mr. Christian Stary (SIEMENS/University of Linz) [AUSTRIA]
    Paper Title: An Open Tool for Automated Testing of Interactive Software (9S2)

    In this paper we introduce IDATG (Integrating Design and Automated Test Case Generation), a specification technique and a tool for automated test case generation when testing industrial interactive applications. It does not only reduce the set of test cases to those that are relevant from the end users' task domain, but also the effort to be spent to specify the corresponding states of the graphical user interface. IDATG supports both the specification and generation of test cases based on an end user task model, as well as test cases oriented towards the complete coverage of the user interface behavior. In both cases the test procedure is based on a particular test process model, and a formal language to represent the user interface. The architecture of the tool is composed of a set of visual editors, a language interpreter, and a test case generator. The interface concept of the components enable the embodiment of the tool into industrial platforms for test case definition and execution. A first cost/benefit analysis indicates a significant reduction of effort for test case specification and test result analysis.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Ms. Cheryl Y. Moore (FedEx Corporation) [USA]
    Paper Title: Testing Policies and Standards (3M1)

    Most test efforts span multiple organizations, groups and/or teams. The major objective of standardizing test processes is to define the policies and procedures to be utilized by all test participants during the testing phase. This document outlines procedures required to accomplish quality testing across various internal organizations and divisions. The FedEx Software Testing Process document addresses the following key issues:

  • Test Planning, Coordination and Management
  • Test Execution
  • Identification of Continuous Improvement Measurements

    Outlined within the document are recommended methods for effective communications, test structures which emphasize roles and responsibilities for test accountability; defined deliverables for documentation of test plans, schedules, cases, and checklists; formal meetings and reviews for communicating status, problem detection and prevention, test and implementation planning; and methods for identification of continuous improvements.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

  • Mr. John D. Musa (Independent Consultant) [USA]
    Tutorial Title: More Reliable, Faster, Cheaper Testing through Software Reliability Engineering (TC)

    This tutorial will quickly, efficiently teach you the basics of how to apply Software Reliability Engineering (SRE) to testing and development to make software more reliable and to develop and test it faster and cheaper.

    SRE is based on four simple, powerful ideas:

  • Set quantitative reliability objectives that balance customer needs for reliability, timely delivery, and cost
  • Track reliability during test
  • Characterize quantitatively how users will employ your product
  • Maximize efficiency of development and test by focusing resources on the most used and/or most critical operations, by realistically reproducing field conditions, and by delivering just enough reliability.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Ms. Anna Newman (Adobe Systems, Inc.) []
    Paper Title: Lessons Learned: Automating Testing Experiences with FrameMaker 5.5 (7A2)

    Adobe's Frame Engineering Group has just completed a 2 year major QA cycle with an extreme focus on automation. Our GUI intensive software released on diverse platforms must maintain a consistency of look and feel as well as functionality, but the resources required to verify this consistency were becoming prohibitively large with each successive release. To maximize our testing effectiveness, Adobe decided to take on the automation of its 4 language, 3 product, 7 platform version FrameMaker 5.5 release. Developing automation at any time for any project is challenging, and Adobe's experiences meeting this challenge, the road blocks encountered and the solutions which were devised, should be of interest to those either planning or implementing an automation effort of their own. Adobe worked to create a reproducible, reliable automated test suite, and we had very ambitious goals, such as total regression of all automated tests on every build and platform. We were ultimately successful, but faced many difficulties along the way, including unexpected design problems, skeptical development engineers, and a morass of conflicting results records. Our solutions to these and other problems enabled us to reach our goals, and we have seen a rise in the quality of our software and of our testing generally. An additional benefit has been improvement in our processes and record keeping, enabling us to move closer to completion of our ISO 9000 goals.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Ms. Patricia O'Reilly (InPower) [USA]
    Paper Title: The Accidental Improvement Opportunity is Knocking... (7M2)

    We have all heard of survival of the fittest. Sometimes this means whoever is the biggest, the strongest. But more and more, it means those who can maneuver, adapt, change. With today's ever increasing competition for scarce resources, accidents need to be more than just accidents. You need to recognize the accidental as an opportunity and use it for process improvement. Then you can leverage what you do to survive to improve the performance of your organization. This is especially effective for those companies at Level 1 and Level 2 SEI behavior.

    The presentation will describe how a company can transform itself by learning to recognize improvement opportunities, gain confidence from them, and begin the transformation into a learning organization. InPower's Product Engineering organization is used as a case study demonstrating how to recognize survival instincts as valid and valuable input into process improvement efforts. This case study is a "work in progress". The study covers how we developed this concept and our experiences to date in what works and what doesn't.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Thomas Ostrand, Dr. Herbert Foster, Dr. Tarak Goradia & Dr. Wojciech Szermer (Siemens Corporate Research) []
    Paper Title: A Visual Test Development Environment for GUI Systems (3T1)

    (Abstract to be supplied)
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Dave Parnas (Communications Research Laboratory, McMaster University) [USA]
    Paper Title: Software Engineering: An Unconsummated Marriage (5P2)

    Most engineers understand very little of the "science of programming" and computer scientists understand very little about what it means to be an Engineer. This talk attempts to explain each group to the other and to describe what each discipline can provide to the next generation. We conclude by describing what true "software enginers" the product of a more interactive marriage, should know.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Rainer Pirker & Mr. Andreas Rudolf (IBM) [AUSTRIA]
    Paper Title: Testing the Software Portfolio of a Bank for Year 2000 Readiness (6M1)

    This is an experience report from a large 'Year 2000' testing project where a large bank in Austria outsourced their Year 2000 testing to IBM. This presentation shows our test strategy, our plan, the measurements we used, what went right and what went wrong, and what is different about testing Year 2000 projects from other software testing. The main focus of this paper is on the implementation and customization of the test concept and process.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Martin Pol (IQUIP Informatica B.V.) []
    Tutorial Title: Test Process Improvement (TJ)

    After a short introduction, the attendees will learn about The Context of testing in the real world and software and test process improvement: why testing, the aims of testing, quality management and testing, what to test, required structure, SDLC and testing, the evolution of testing, the challenges for testing, need of improvement, need of a dedicated test improvement model, other available models, the model requirements, etc.

    The second part of the tutorial is about the Test Process Improvement Model. The TPI (tm) model will be explained in detail: the scope, the characteristics, the key areas, the requirements (checkpoints) for the different levels of maturity, the test maturity matrix, the assessment, the relation between the key areas and how to select priorities, the improvement suggestions, etc.

    To be successful with any improvement activity an adequate Management of Change approach is required. The third part of the tutorial is about The Application of the Model. The main steps of changing will be taught: how to create awareness, establish goals and scope for change, the assessment process, selection and planning of improvement actions, implementation and evaluation. Delegates will learn about related subjects such as the use of metrics, the requirements for the change team, the human aspects, how to deal with resistance and finally some do's and don'ts will complete the tutorial.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Robert Poston (AONIX, Inc.) [USA]
    Paper Title: Making Test Cases from Use Cases Automatically (7Q)

    Use cases are graphical notations that were created to help document and test system behavior. Ivar Jacobson introduced use cases in 1992, and they are now defined in the new UML (Unified Modeling Language). However, use cases fall short of their intended purpose, because they do not provide enough information for test case or script generation. Testers must go outside use cases to find enough information to create test cases.

    However, the UML is extendable, and standard use cases can be extended with additional information to become test-ready use cases. Test-ready use cases contain sufficient information for automatic test case creation. This paper describes test-ready extensions to use cases and the process of automatic test case generation.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Mike Powers (ST Labs, Inc.) [USA]
    Paper Title: Testing in a Year 2000 Project (7S2)

    Minimize the risks in your Year 2000 project by choosing the right test techniques before, during and after conversion. The right strategy for a Year 2000 project is essential for success, and testing can help make sure that you will meet your tight schedule.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Anil Rao (Hewlett-Packard Company) [USA]
    Paper Title: A Structured Framework for Designing Kernel Reliability Tests (3T2)

    This paper investigates the issues involved in designing reliability tests for an operating system kernel. The current techniques used for developing these tests are discussed and their limitations identified. In order to overcome these limitations a new methodology is proposed which is based on a structured framework. This design strategy leads to more robust test programs and it also eases the development and maintenance efforts. It is also shown how the new framework was used in designing the tests for two representative HP-UX kernel sub-systems.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. David J. Brown & Dr. Jarrett Rosenberg (Sun Microsystems) [USA]
    Paper Title: Static Checking of Application Binaries for Cross-Release Stability (6T1)

    A common problem in evolving computing environments is ensuring that applications continue to run despite changes to the environment. We describe a methodology for checking application binaries to determine if they use unstable system interfaces which would put them at risk when the operating system is later upgraded.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Linda Rosenberg & Mr. Larry Hyatt (Unisys/SATC CSFC NASA) [USA]
    Tutorial Title: Metrics for Quality Assurance and Risk Assessment (TA)

    This presentation will:

  • Provide project managers and software developers with the knowledge to institute an affordable metrics program that will evaluate the quality of their project's products and to help them identify and track project risks.
  • Demonstrate a model for metrics programs and a core set of metrics being applied within NASA, including metrics for object-oriented development, re-engineering and COTS applications
  • Discuss metric program costs, benefits and techniques for starting a program
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Dr. Linda Rosenberg, Mr. Larry Hyatt, Mr. T. Hammer, Ms. L. Huffman & Mr. W. Wilson (Unisys/SATC CSFC NASA) [USA]
    Paper Title: Testing Metrics for Requirement Quality (2T2)

  • Requirement metric program guidelines including requirement quality attributes
  • Identify metrics available in the requirements phases that assist in the verification and validation of the requirements and test plans
  • Demonstrate how NASA has applied these metrics to improve their testing processes and hence product quality
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Ms. Johanna Rothman & Mr. Brian Lawrence () [USA]
    Panel Title: The Role of the Test Manager (4P)

    There is not universal agreement on the role of the Test manager/SQA Manager/Technical lead in software product development. In fact, there is not even agreement on the name. When there is not sufficient agreement, people make assumptions of the responsibilities of this individual. This panel proposal is intended to surface assumptions, clarify potential roles, and discuss the effects of choosing responsibilties.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Raymond V. Sandfoss & Mr. Steven A. Meyer (AT&T Laboratories) [USA]
    Paper Title: The Impact of OOT, Client/Server, and Distributed Computing on SRE Practices (3A2)

    This talk explores the field of SRE and techniques that can be used for the latest brand of computing, where "the Network is the System". Some topics covered are : Applying Operational Profile techniques to Use-Case development, Measuring components of a Distributed Environment, and Applying the SRE methodology to Distributed Solutions
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Ms. Carla Schroer (Sun Microsystems) [USA]
    Paper Title: Java(tm) Platform Compatibility Testing (9T2)

    This session covers the Java Compatibility Kit (JCK) and how it is used to certify Java platform implementations. The focus is on the details behind the JCK: what is it for? what does it cover? what techniques are used? what tools have we built?
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Jeff Singer & Ms. Cindy Friedman (Ensemble Partners, Inc.) [USA]
    Paper Title: The Software End Game - Accurate Prediction of Time Product Completion (9M2)

    A large majority of commercial software firms have difficulty in deciding when the quality level of a product is adequate for general distribution to the market. This is largely due to a lack of in-process quality metrics in the development organization. As a result, the process of deciding whether or not to ship, is based on limited data that at best, give only a very rough indication of the state of the product. In this paper, we present:

  • A technique developed in our consulting practice for predicting the time of completion of testing defined by target quality level.
  • The historical data required to apply the technique consisting of program size, number of total defects found and time of defect discovery.
  • A case study from our consulting practice demonstrating the application of the technique and the results that were obtained.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Mr. Lincoln Spector (Computer Journalist, Columnist, Humorist) [USA]
    Paper Title: Why are Computers So Damned Funny? (8M1)

    Why are computers so damned funny? Where is Microsoft taking us? Who is running Apple? How can we create easy-to-use software for operating systems that crash when we look at them hard? Lincoln Spector, author of the internationally-published humor column Gigglebytes, will make nonsense of it all.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Keith Stobie (BEA Systems, Inc.) [USA]
    Paper Title: TestWare Engineering (9A2)

    This talk covers several areas of testware design. It discusses several areas of the research literature including theoretical and empirical papers and what it means to the practicing testware engineer.

    Questions addressed include:

  • When is partition testing better than stochastic testing?
  • What does coverage tell us about test sets?
  • When can tests giving duplicate coverage be eliminated?
  • Optimal way to do combination testing?
  • What can be done to avoid huge numbers of tests to achieve reasonable MTBF?
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM
  • Mr. Ondrej Such (Microsoft Corporation) [USA]
    Paper Title: Applications of Stochastic Asynchronous Programming Technique to Procedure Testing (4A2)

    (Abstract to be supplied)
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Pascale Thevenod-Fosse & Dr. Helene Waeselynck (LAAS - CNRS) []
    Paper Title: Software Statistical Testing Based on Structural and Functional Criteria (7T1)

    Statistical testing is based on a probabilistic generation of test patterns: structural or functional criteria serve as guides for defining an input profile and a test size. The method is intended to compensate for the imperfect connection of criteria with software faults, and should not be confused with random testing, a blind approach that uses a uniform profile over the input domain. After a brief description of the approach, the paper is focused on experimental results involving safety-critical software from various application domains: avionics, civil and military nuclear field.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. Otto Vinter (Bruel & Kjaer) [DENMARK]
    Paper Title: Improved Requirements Engineering Based on Defect Analysis (2A1)

    At QW'96 and '97 Bruel & Kjaer reported the experiences of a software process improvement (SPI) project where we demonstrated that the introduction of static and dynamic analysis in our software development process had a significant impact on the quality of our products.

    The basis for this project was a thorough analysis of error reports from previous projects which showed the need to perform a more systematic unit test of our products. However, the analysis also showed that the major cause of bugs stemmed from requirements related issues.

    We are currently conducting another SPI project where we analyze the requirements related bugs in order to find and introduce effective prevention techniques in our requirements engineering process with the objective of reducing the number of requirements related error reports by 50%.

    This presentation will cover the analysis results, a set of effective prevention techniques, and also the practical experiences using some of these techniques on real-life development projects.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Ms. Anneliese vonMayrhauser, Mr. Andre Bai, Mr. Tom Chen, Mr. Amjad Hajjar & Mr. Charles Anderson (Colorado State University) [USA]
    Paper Title: Fast Antirandom (FAR) Test Generation to Improve Code Coverage (4T1)

    Anti-random testing has proved useful in a series of empirical evaluations. The basic premise of anti-random testing is to chose new test vectors that are as far away from existing test inputs as possible. The distance measure is Hamming or Cartesian Distance. Unfortunately, this method essentially requires enumeration of the input space and computation of each input vector. This prevents scale-up to large test sets and/or long input vectors.

    We present and empirically evaluate a technique to generate anti-random vectors that are computationally feasible for large input vectors and long sequences of tests. We also show how this fast anti-random test generation (FAR) can consider retained state (i. e. effects of subsequent inputs on each other). We evaluate effectiveness using branch coverage as the testing criterion.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Mr. William H. Warren (Independent Consultant) [USA]
    Paper Title: The Business Cost of Defective Software (4M1)

    The costs of defective software are having a significant impact on the profit structures of many businesses. It is really necessary to start with an assessment of what are the cost mechanics of a given firm, and an understanding of the impact of defective software on this structure before it is possible to create the appropriate corrective action plans.

    The model we will review is useful in structuring one's thinking about the cost of defective software that is shipped with a firm's product. This is often called "embedded software", or it could be the entire product line for a software only supplier. There are two types of costs that need to be estimated, the operational costs associated with the various departments in a firm, and the non-measurable costs that are most often "lost revenue" situations.

    The primary goal of this presentation is to provide a methodology on how you can create a measurement of the amount of cost that is being incurred due to defective software escaping into the company's production and customer environment.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Elaine J. Weyuker & Dr. Alberto Avritzer (AT&T Laboratories) [USA]
    Paper Title: Facilitating the Enforcement of Quality of Service Objectives by Using Software Testing Artifacts (7T2)

    We present four algorithms designed to enforce different Quality of Service criteria for shared distributed objects that have been tested using a previously-introduced test generation algorithm. Empirical assessments of these algorithms for three large industrial telecommunications systems are also presented.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM

    Dr. Karl E. Wiegers (Process Impact) []
    Paper Title: Software Process Improvement: Ten Traps to Avoid (8M2)

    Even well-planned software process improvement initiatives can be derailed by one of the many risks that threaten such programs. This presentation describes ten traps that can undermine any process improvement program. The symptoms of each trap are described, along with several suggested strategies for preventing and dealing with the trap.
    JUMP TO TOP OF PAGE
    RETURN TO QW'98 PROGRAM