Managing the Test Activities
Managing the Test Activities
Complete Chapter 5 study notes covering test planning, risk management, monitoring and control, configuration management, and defect management.
Test Planning
FL-5.1.1–FL-5.1.7 · Test plans, planning contribution, entry/exit criteria, estimation, prioritization, test pyramid and testing quadrants.
Purpose of planning
Define objectives, means, schedule, risks and the overall test approach. Planning is ongoing and is updated as feedback and risks change.
Typical test-plan content
Context, assumptions and limitations, stakeholders, communication, product/project risks, test approach and schedule.
Strategy → approach
The test approach implements the chosen strategy for the project. Strategies may be analytical, model-based, methodical, process-compliant, directed, regression-averse or reactive.
5.1.1 Purpose and Content of a Test Plan
A test plan describes the test project's objectives, the means needed to achieve them and the schedule of test activities. Large projects may have a master plan plus plans for specific test levels or test types.
| Planning area | Examples |
|---|---|
| Context | Scope, objectives, limitations, test basis |
| Stakeholders | Roles, responsibilities, influence, training needs |
| Communication | Types, frequency and document templates |
| Risks | Product risks and project risks |
| Approach | Levels, types, techniques, work products, entry/exit criteria, metrics, data and environments |
| Schedule | Timing and execution order |
5.1.2 Tester's Contribution to Iteration and Release Planning
| Release planning | Iteration planning |
|---|---|
| Define testable user stories and acceptance criteria | Detailed risk analysis of selected stories |
| Participate in project/product risk analysis | Determine testability |
| Estimate testing effort for user stories | Co-create acceptance tests |
| Define necessary test levels | Split stories into tasks and estimate testing tasks |
| Plan testing for the release | Identify functional/non-functional aspects and support automation |
5.1.3 Entry Criteria and Exit Criteria
Ready to start?
Done enough?
| Entry criteria | Exit criteria |
|---|---|
| Testable requirements/stories/models | Scheduled tests completed |
| Required test items available | Required coverage achieved |
| Environment, tools and data ready | Open-defect limit not exceeded |
| Initial quality adequate, e.g. smoke tests pass | Required defect density/reliability achieved |
5.1.4 Estimation Techniques
Test effort is the amount of test-related work required to achieve the test objectives and is commonly expressed in person-time such as person-days.
Ratio-based
Use historical ratios between activities to estimate a similar project.
Extrapolation
Use measurements from earlier iterations or completed work to predict future effort.
Wideband Delphi / Planning Poker
Experts estimate independently and converge through discussion.
Three-point estimation
Use optimistic, most likely and pessimistic estimates to account for uncertainty.
5.1.5 Test Case Prioritization
A test execution schedule must account for priorities, dependencies, confirmation/regression needs and an effective execution order. Common prioritization bases include risk, coverage and additional coverage. Dependencies can override a desired priority order.
5.1.6 Test Pyramid
Higher layers generally contain fewer, larger, slower and more expensive tests; lower layers contain many small, isolated and fast tests. The exact shape depends on project context.
5.1.7 Testing Quadrants
| Supporting the team | Critiquing the product | |
|---|---|---|
| Technology-facing | Q1: component tests; highly automated; CI | Q4: most non-functional tests; often automated |
| Business-facing | Q2: functional tests, examples, stories, prototypes, acceptance criteria | Q3: exploratory, scenario/use-case, usability, UAT, alpha/beta |
Risk Management
FL-5.2.1–FL-5.2.4 · Risk level, project/product risks, product risk analysis and product risk control.
Risk-based testing uses risk analysis and control to manage, select and prioritize testing activities and resources.
5.2.1 Risk Definition and Risk Attributes
Likelihood and impact are separate attributes. Risk assessment may be quantitative, qualitative using a risk matrix, or mixed using ranges.
5.2.2 Project Risks and Product Risks
| Project risks | Product risks |
|---|---|
| Threaten project success: schedule, budget, scope, people, suppliers, tools, environment or process. | Threaten product quality: missing functionality, wrong calculations, crashes, poor architecture/performance/usability or security vulnerabilities. |
| Typical consequence: delay, cost increase, incomplete iteration. | Typical consequence: dissatisfaction, revenue loss, maintenance cost, harm or penalties. |
5.2.3 Product Risk Analysis
Generate a comprehensive list using brainstorming, workshops, Delphi/expert assessment, interviews, checklists, lessons learned, cause-and-effect diagrams and templates.
Categorize risks, estimate likelihood and impact, determine risk level, prioritize, and propose responses.
Risk analysis influences test levels/types, scope, priorities, techniques, effort and other mitigation measures.
5.2.4 Product Risk Control
| Response | Meaning |
|---|---|
| Risk acceptance | Accept a low-level risk rather than spend disproportionate effort on it. |
| Risk transfer | Transfer consequences to another party, e.g. insurance. |
| Contingency plan | Prepare actions in advance for when the risk occurs. |
| Mitigation by testing | Use reviews, static analysis, appropriate techniques/coverage, independence, prioritization and regression testing. |
Risk monitoring continuously checks whether mitigation is effective, reports residual risk and captures new or changed risks.
Test Monitoring, Test Control, and Test Completion
FL-5.3.1–FL-5.3.3 · Metrics, reports and communication of test status.
5.3.1 Metrics Used in Testing
| Metric family | Examples |
|---|---|
| Project | Task completion, resource use, effort, milestones |
| Test case | Implementation progress, run/not run, pass/fail, execution time |
| Product quality | Availability, response time, mean time to failure |
| Defect | Found/repaired, priorities, density, frequency, confirmation success |
| Risk | Residual risk level, risk priority |
| Coverage | Requirements, stories, criteria, conditions, code, risks |
| Cost | Testing cost, quality cost, execution/repair cost |
5.3.2 Purpose, Content, and Audience for Test Reports
| Progress report | Completion report |
|---|---|
| Produced during testing. | Produced at completion milestones. |
| Current status, plan progress and tests scheduled for next reporting period. | Evaluation against exit criteria, deviations, product quality, defects and residual risks. |
Reports should be tailored to their audience; different stakeholders need different levels of detail.
5.3.3 Communicating the Status of Testing
Communication can be verbal, dashboards, electronic channels, online documentation or formal reports. The appropriate mix depends on stakeholder concerns, organizational strategy, regulation and team context.
Configuration Management
FL-5.4.1 (K2) Summarize how configuration management supports testing.
Configuration management maintains the integrity of the component/system, testware and the relationships among them throughout the lifecycle.
Identify
Know which test objects, test items and testware are configuration items.
Version
Control versions and track changes.
Trace
Link related software, documentation, environments and testware.
Reconstruct
Recreate the exact components and test assets used for a particular release.
Defect Management
FL-5.5.1 (K3) Prepare a defect report.
Defects should be logged, investigated and tracked from detection and classification until resolution. Reports also support communication, quality assessment, decision-making, risk identification and process improvement.
Useful defect-report information
| Unique identifier | Title and brief summary | Date and author |
| Item/product/configuration identification | Lifecycle phase | Steps/details enabling reproduction |
| Actual and expected results | Priority / urgency / status | Logs, dumps, screenshots or recordings |
| Nonconformity description | Conclusions/recommendations | History and references to related test cases |
Important: “Fixed” does not mean “closed.” From the tester's perspective, a fix should be retested before the defect is closed.
Interactive Explorer
Key Takeaways
Plan continuously
The test plan evolves as feedback, risks and project conditions change.
Use risk to focus effort
Likelihood and impact determine risk level; testing should reduce residual product risk.
Measure what matters
Metrics support monitoring and control, but reports should remain focused and audience-specific.
Preserve traceability
Configuration management makes releases and their test assets reproducible.
Report defects reproducibly
A useful defect report gives enough information to understand, reproduce, prioritize and track the anomaly.
Knowledge Check
Answer as many questions as you like, then use the single button after Question 5.16. Unanswered questions are allowed.
Which of the following is NOT part of the test plan?
Which of the following is done by the tester during release planning?
Consider the following entry and exit criteria:
i. Availability of testers.
ii. No open critical defects.
iii. 70% statement coverage achieved in component testing.
iv. All smoke tests performed before system testing have passed.
Which are entry criteria and which are exit criteria?
The team estimates each iteration as the average effort of the previous three iterations. The first three iterations required 12, 15, and 18 person-days. What is the estimate for the FIFTH iteration?
Using additional feature coverage, which test case is executed THIRD?
TC1: A, B, C, F
TC2: D
TC3: A, F, G
TC4: E
TC5: D, G
What does the test pyramid model describe?
In which test quadrant is component testing located?
The likelihood of a system performance risk was estimated as “very high.” What can be said about the impact of this risk?
Which of the following is an example of a consequence of a project risk?
A mobile banking project identifies two risks: an overly complicated transfer interface, especially for seniors; and a transfer mechanism that executes payments late when they fall on Saturday or Sunday. What are the most reasonable mitigation actions?
After release, the software producer buys insurance in case a malfunction causes users to lose their health. What kind of risk response is this?
Which of the following is NOT a metric used for testing?
Which of the following will NOT normally be included in a test completion report?
Which of the following is the BEST form of communicating the status of testing?
A team can reconstruct all component and testware versions used to build a customer's software release from its version number. Which process enables this?
A login defect report contains a title, brief summary, product version, risk level and priority. What VALUABLE information is missing?