The Foundations of Software Testing
1️⃣ Quality and the V-Model Life Cycle Process
2️⃣ Process Overview
3️⃣ Test Overview
1️⃣ Quality and the V-Model Life Cycle Process
1. Definition of Quality
"Quality" is a term we use all the time; we say a product or service is "high quality" or "low quality." But how exactly is quality defined?
Definitions by Leading Scholars
W. E. Deming (Father of Quality): "Quality is meeting the needs of customers."
J. M. Juran: "Quality is fitness for use."
P. B. Crosby: "Quality is conformance to requirements."
Synthesizing these definitions, quality is determined by how well a product or service satisfies customer or user needs - in other words, how closely it conforms and how fit it is for its intended purpose.
Definition of Quality: The totality of characteristics of a product or service that bear on its ability to satisfy stated and implied requirements.
Stated Requirements vs. Implied Requirements
Stated requirements: Requirements that are documented or formally and explicitly defined.
Implied requirements: Requirements that are not documented but are implicitly expected by the user. Customers often cannot fully articulate everything they need, and these taken-for-granted, hidden expectations are what we call implied requirements.
Therefore, a development organization must uncover and analyze not only stated requirements but especially implied requirements, and reflect them in the product or service - only then can the team deliver what customers truly want.
2. Two Perspectives on Quality
What is ultimately delivered to the customer is the final product, but evaluating quality does not mean looking at the end deliverable alone. Quality must be viewed from two perspectives.
1) Process Quality
This refers to the quality of the entire workflow, including development, maintenance, and the work products produced at each stage. The higher the process quality, the more positive its impact on product quality. Process quality is based on the PDCA cycle.
Plan: Establish a plan before starting work.
Do: Execute the work according to the plan.
Check: Regularly monitor whether the work is on track.
Act: Continuously improve the process.
Organizations that follow the PDCA cycle demonstrate strong process quality, ensuring that all work products are managed systematically.
2) Product Quality
Work products generated through the process must ensure traceability and consistency. Only when the final software or system satisfies the customer's requirements can we say the product quality is good.
Key Takeaway: It is not just the final deliverable that matters. Process quality must be raised first — this improves the quality of intermediate work products at each stage, which in turn elevates the quality of the final deliverable.
[Reference] Quality Management System Based on PDCA
A quality management system takes inputs such as customer requirements and the needs and expectations of relevant stakeholders, and operates across departments — including planning, marketing, sales, R&D, development, testing, and production — all working in accordance with the PDCA cycle. Even a single coding task follows the Plan → Do → Check → Act cycle. Through this, organizations continuously improve customer satisfaction, QMS performance, and the quality of products and services.
[Reference] Relationship Between Process Quality and Product Quality: The BMW Case Study
BMW divided its component suppliers into two groups — those with high process quality and those with low process quality — and analyzed the relationship between process quality and product quality. Product quality was measured by the number of defects discovered prior to SOP (Start of Production).
High process quality organization:
16 months before SOP: 50% of total defects identified
11 months before SOP: 90% of total defects identified → 11 months of buffer time remaining
Low process quality organization:
8 months before SOP: 50% of total defects identified
2 months before SOP: 90% of total defects identified → 10% of defects still unresolved just before production begins
The gap between the two groups in reaching the 90% defect detection milestone is a striking 9 months. This case study clearly demonstrates that process quality has an enormous impact on product quality — and this applies not only to the automotive industry but across all industries.
3. V-Model Life Cycle Process
The V-Model is a software development life cycle (SDLC) model in which the development activities on the left side correspond directly to the testing activities on the right side. It is a model that places strong emphasis on Verification and Validation (V&V), and is also named after the initials of these two concepts.
[Reference] Concepts of Verification and Validation
Verification: Confirming that the product conforms to its specifications, requirements, and design specs. "Are we building the product right?"
Validation: Confirming that the product meets the user's intended use and purpose. "Are we building the right product?"
Example: Testing an electric vehicle against its technical specifications is Verification; evaluating it from the actual end user's perspective is Validation.
[Reference] V&V Techniques
V&V techniques are broadly divided into Static methods and Dynamic methods.
Static methods: Techniques for finding defects without executing the program.
Review: Document and code reviews (e.g., peer review, walkthrough, inspection)
Analysis: Static code analysis, formal methods, etc.
Dynamic methods: Techniques for finding defects by actually executing the program.
- Testing: Black-box testing, white-box testing, etc.
Development–Test Correspondence in the V-Model
Customer/User Requirements ↔ Acceptance Testing (AT)
Requirements Analysis ↔ System Testing (ST)
Architectural Design ↔ Integration Testing (IT)
Detailed Design ↔ Unit Testing (UT)
Implementation (coding) → Execute tests sequentially, starting from Unit Testing
It is important to note that this correspondence does not mean "run the tests as soon as each development phase is complete." Rather, as each development phase progresses, the corresponding test planning activities — including test procedures, test approaches, environment setup, and test case development — are carried out in parallel. The actual test execution begins after the code is implemented and proceeds in the order: Unit Testing → Integration Testing → System Testing → Acceptance Testing.
Advantages of the V-Model
Improved quality of work products: By applying static methods such as reviews and analysis, defects in requirements documents, architecture documents, and code can be identified before execution, improving the overall quality of all work products.
Shorter schedule through parallel development and test planning: Running development and test planning in parallel fosters close communication between teams. As the quality of upstream work products improves, the workload in downstream phases is reduced — leading to an overall shorter project schedule.
2️⃣ Process Overview
1. Definition and Role of a Process
In general terms, a process defines the steps, sequences, and procedures for carrying out a piece of work. That is the dictionary definition. But when we look at the role a process actually plays, we can see something more specific.
A process exists so that we can systematically create products; such as vehicles, systems, components, or software - that satisfy customer requirements, including functional requirements, non-functional requirements, and constraints. What does it take to do work well? Work proceeds systematically when procedures and methods, tools and equipment, and personnel are properly integrated. A process can therefore be defined as "the means of integrating procedures/methods, tools/equipment, and personnel in order to build a product that satisfies customer requirements."
Consider a waste sorting facility as an example. When a garbage truck arrives and workers need to sort plastics, aluminum, and paper efficiently, they need defined procedures, equipment set up for efficient sorting, and clearly assigned personnel. When all of these elements work together naturally, the sorting operation runs smoothly. In the same way, a process acts as the glue that integrates procedures, methods, tools, equipment, and people so that work can be done well. The role of a process is significant.
2. Defining a Plan Means Defining a Process
Defining a process and creating a plan are essentially the same thing. Planning is not simply about writing a to-do list — it includes defining the process itself. In other words, a plan must define more than just a schedule. A process binds together procedures, methods, tools, equipment, systems, and personnel. The same applies when planning. To perform testing effectively, you need applicable methods and procedures, systems and equipment for efficiency, and personnel assigned to the right roles. Only when all of these elements are properly in place does the work proceed smoothly.
In short, planning is not merely a to-do list — it is the act of defining a process, and a process-driven plan is what truly matters.
[Reference] How to Define a Process: ETVX
ETVX is a framework for systematically defining the activities to be performed at each stage of a process. The acronym stands for:
Entry Criteria: The conditions that must be met before a task can begin — the entry gate for starting the work.
Task: The detailed activities to be performed and the steps in which they are carried out.
Verification: The criteria for verifying that the work in progress is being done correctly — a mid-process quality check.
eXit Criteria: The conditions that must be satisfied for the work to be considered complete — the final completion gate.
ETVX can be applied to each phase of software development, including requirements development, design/implementation, and testing. Here is an example of how ETVX is applied to the software requirements development process in practice.
Process: Software Requirements Development
Purpose: Analyze customer and system requirements to develop software requirements.
Applying ETVX:
Entry Criteria: Customer requirements and system requirements analysis results are delivered as inputs.
Task: Analyze, specify, and review the software requirements.
Verification: Check that customer and system requirements are properly reflected in the software requirements. Confirm that an objective review of the software requirements has been conducted.
eXit Criteria: System requirements must be fully converted into software requirements, and the software requirements review must be completed with no outstanding issues.
In addition to the ETVX steps, a process definition also includes Tools, Methods, and Roles — because a process encompasses not just procedures but also tools, equipment, and personnel.
Tools: Requirements modeling tools, requirements management systems.
Methods: Stakeholder interviews (using checklists), inspection-based reviews.
Roles: Requirements analysis → performed by the requirements engineer; requirements specification → performed by the requirements engineer; requirements review → performed by reviewers such as the architect and tester.
By defining procedures, methods, tools, equipment, and personnel within the ETVX framework, teams can clearly understand each process and apply the right approach at every stage. This framework is widely used in industry and is well worth knowing.
3️⃣ Test Overview
1. Definition of Testing
Let's look at some of the most widely referenced definitions of software testing.
Myers defines testing as "the process of executing a program with the intent of finding defects." This definition emphasizes that the fundamental purpose of testing is defect detection.
Craig and Jaskiel define testing as "a lifecycle process that engineers, uses, and maintains testware in order to measure and improve the quality of the software being tested." Here, testware refers to the various work products and tools produced during the planning and design of testing activities. In other words, this definition frames testing as the systematic and engineering-based application of testware to measure and improve how well a software product satisfies both stated and implied customer requirements.
IEEE Std 829 defines testing as "the process of analyzing a software item to detect the differences between existing and required conditions and to evaluate the features of the software item." This definition focuses on identifying the gap between the expected behavior of a system and its actual behavior under defect, error, or bug conditions.
While these definitions vary in focus, they share a common goal: to find as many defects and bugs as possible — before the software is released. Once code is complete, it undergoes a series of test levels including Unit Testing, Integration Testing, System Testing, and Acceptance Testing, with various techniques applied at each level. Since defects discovered after deployment or production cause far greater quality issues, the priority is early defect detection across multiple perspectives before release.
2. The Test Process from a PDCA Perspective
As discussed in the context of the V-Model, actual test execution only becomes possible once the code has been implemented. However, the preparation that takes place before execution is equally critical. The test process as a whole follows the PDCA cycle.
Plan — Test Planning
The overall test plan is established and the features to be tested are identified. This phase covers planning activities such as defining the test schedule, test scope, test items, test strategy, and test environment. In the V-Model, test planning runs in parallel with the development phases — customer/user requirements, requirements analysis, architectural design, and detailed design — with corresponding test plans being prepared alongside each. Close communication and collaboration between the development team (left side of the V) and the test team (right side of the V) is the core concept of the V-Model.
Plan — Test Design
Based on the test plan, test cases are developed and test procedures are defined. The test environment is also prepared to replicate real-world usage conditions. For example, in automotive software development, this would mean setting up an environment that reflects actual road conditions.
Do — Test Execution
The prepared test cases are executed and the actual results are compared against expected results to produce a pass/fail verdict.
Check / Act — Test Evaluation and Improvement
Test results are analyzed and evaluated. Progress is monitored and controlled against the test plan, corrective actions are taken when issues arise, and the test process itself is continuously improved.
In summary, testing is not simply the act of execution (Do). The full test process encompasses the entire PDCA cycle — planning, design, execution, and evaluation/improvement — and this is an important point to keep in mind.
[Reference] ISO/IEC/IEEE 29119 Software Testing Standard
An internationally recognized standard for software testing exists in the form of ISO/IEC/IEEE 29119. The standard is structured as follows:
Part 1 — Concepts and Vocabulary: Defines common concepts and terminology used in software testing.
Part 2 — Test Processes: Defines the framework for software test processes.
Part 3 — Test Documentation: Defines requirements for test work products and documentation.
Part 4 — Test Techniques: Defines test design techniques applicable to software testing.
Part 5 — Keyword-Driven Testing: Covers test automation through the use of keyword-driven test scripts.
The standard defines the test process using a multi-layer architecture consisting of three levels:
1) Organizational Test Process
At the organizational level, the Test Policy and Test Strategy are established. These are then passed down to the project level, where they serve as the governing framework for all test activities.
2) Test Management Process
This process operates at the project level and manages testing across different test levels and types — including Unit, Integration, Performance, and Security Testing. It defines a series of activities: test planning, test monitoring and control, and test completion.
3) Dynamic Test Process
This is the process by which testing is actually carried out. It consists of a series of activities including test design and implementation, test environment setup and maintenance, test execution, and test results reporting.
This standard provides an internationally agreed-upon framework for establishing and operating a systematic test process, and is widely referenced in industry practice.
