Software Engineering / Lecture 03 01 / 09

What Testing Is
And What Testing
Does.

Definitions · Goals · Key Terms

A systematic and methodical examination.

◉ EST. 2024 · DEPT. OF CS

Introduction · definition 02 / 09

Definition of software testing

Attempting to show that it does not fulfil its intended purpose.

Testing is the systematic and methodical examination of a work product using a variety of techniques, with the express intention of attempting to show that it does not fulfil its desired or intended purpose.

This is undertaken in an environment that represents most nearly that which will be used in live operation.

Like other definitions of testing, it is not perfect. But it points the way.

Systematic: It has to be planned.

Methodical: It is a process.

Work product: It is not just code that is examined.

Variety of techniques: Different approaches are required.

Principle reasons · the aims 03 / 09

What does testing aim to do?

Not all testing has the same aim.

01To examine work products (requirements, user stories, design, code).

02To check if all the requirements have been satisfied.

03To see whether the item under test is complete and works as expected.

04To instil confidence in the quality of the item under test.

05To prevent defects – stopping them happening in the first place.

06To find failures and prevent these reaching the production version.

07To give sufficient information to enable decision makers to release.

08To reduce the level of risk of inadequate software quality.

09To comply with contractual, legal or regulatory requirements.

Key terms · ISTQB glossary 04 / 09

The foundation of test analysis and design

Four terms that define the test process.

Term 01

Test basis

The body of knowledge used as the basis for test analysis and design. Usually written down in requirements or interface definitions. It is the starting point for what to test.

Term 02

Test condition

An aspect of the test basis that is relevant to achieve specific test objectives. Descriptions of what should happen under certain circumstances as a consequence of previous actions.

Term 03

Test case

A set of preconditions, inputs, actions, expected results and postconditions, developed based on test conditions. Describes specific values to enter and the expected outcome.

Term 04

Test procedure

A sequence of test cases in execution order, and any associated actions required to set up initial pre-conditions and wrap up activities post execution.

Testing vs. debugging · distinct roles 05 / 09

Different kinds of activity, both very important

Testing finds. Debugging fixes.

Debugging

Identify. Correct. Check.

The process developers go through to identify the cause of bugs or defects in code and undertake corrections. Ideally, some check of the correction is made to ensure other areas were not inadvertently affected.

Testing

Explore. Report. Verify.

A systematic exploration of a component or system with the main aim of finding and reporting defects. Testing does not include correction—it passes defects to developers. It ensures changes are checked for system-wide effects.

Defects · effects · root causes 06 / 09

Distinguishing the 'why' from the 'what happened'

Similar defects can have completely different root causes.

STAGE / 01

Root Cause

Why it came about. An incorrect understanding by the business lead of how long-term interest rates are to be calculated.

STAGE / 02

The Defect

What is wrong. An incorrect calculation of interest rates in the system logic.

STAGE / 03

The Effect

What happened. An angry customer who is charged too much money on their account.

Testing and quality assurance · hierarchy 07 / 09

Where testing fits in

Testing is part of Quality Control.

Testing is part of Quality Control, which itself is part of Quality Management. Quality Assurance is about making sure processes are undertaken correctly to increase the likelihood of a better end product.

Quality Control is about seeing whether the desired level of quality is being achieved. Checking quality can be undertaken in ways other than testing (e.g. questionnaires).

Quality Management

Quality Assurance (QA)

Making sure processes are undertaken correctly. If processes are right, the end product will likely be better.

Quality Control (QC)

Checking whether the desired level of quality is achieved. Testing is a component of QC.

Static vs. dynamic testing 08 / 09

The discipline of software testing encompasses both

Test as early as possible.

Static Testing

No code execution.

Failures often begin with a human error in a document. Static testing (like reviews) removes ambiguities and faults from specifications. Errors are much cheaper to fix than defects or failures.

Dynamic Testing

Code exercised.

The kind of testing that exercises the program under test with test data. We speak of test execution in this context. It checks the behaviour of the running software.

Process & techniques · conclusion 09 / 09

Effective testing requires process and proven techniques.

A good test is one that finds a defect.

A test that finds no defect has consumed resources but added no value; a test that finds a defect has created an opportunity to improve the quality of the product. We use well-proven test design techniques and general principles to ensure tests are effective.