The intent of Verification and Validation (V&V) is to maintain ICT software system quality throughout the ICT project lifecycle, where V&V:
(a) ensures quality is built into the ICT project from the beginning of the ICT acquisition process,
(b) relates to catching and correcting defects earlier in the development lifecycle in order to create a more robust, high-quality ICT software system,
(c) ensures software or system quality is not treated as an afterthought or relegated to the end of the lifecycle.
This section explains what V&V are and how this terminology is used within this IV&V Handbook.
2.2.1 Definition of V&V
To fully understand the services of an IV&V Provider, a background definition is needed for Verification & Validation (V&V)[1]. Figure 2.2 below summarizes the definition.
Per IEEE, Verification is defined as follows:
(a) Is the practice of verifying documents, designs, software code and program.
(b) Verification checks whether the software conforms to specifications through testing to the specifications to check whether the system was built right.
(c) It can catch defects that validation may not catch and is internally-focused on quality control.
(d) The goal of verification is to demonstrate, inspect, and analyse the traceability to make sure no requirements are missed in the design.
(e) It answers the question: Did we get what we asked for?
Per IEEE, Validation is defined as follows:
(a) It checks whether software meets the customer’s expectations and requirements.
(b) Validation can catch defects that verification cannot catch, e.g., is the product suitable for use in the field with an example of failure being an ICT-driven handheld device that weighs too much to physically carry.
(c) Typically, validation examines the actual product and is performed by Quality Assurance (QA) team to ensure that the software is tested to the business reality to answer the question, are we building the right system?
(d) Validation is externally-focused to examine suitability and effectiveness through end-to-end testing with practical exercises.
(e) The goal is to ensure the product will function to meet the requirements and to assess the question: Did we get what we wanted?
Adhering to V&V standards would lead to the following benefits.
(a) Provide objective ICT system development and risk assessment appraisals.
(b) Adherence to prescribed governing systems development and security assessment management.
(c) Increased visibility into development activities.
(d) Increased visibility into requirements and design phases.
(e) Early problem identification and remediation, as demonstrated by Figure 2.3 below.
Figure 2.3: Cost of Correcting a Defect
(f) Reduced risk with ICT systems development.
(g) Reduce security vulnerabilities.
(h) Adherence to the ICT Development Lifecycles.
(i) Improved system maintainability, reliability, and integrity.
2.2.2 Definition of IV&V
IEEE Std 1012's definitions of Independent Verification and Validation (IV&V), is:
(a) verification and validation performed by an organization that is technically, managerially, and financially independent of the development organization. Figure 2.4 below summarizes the definition.
Figure 2.4: Definition of IV&V
In the context of this IV&V Handbook, the independent organization:
(a) is a third-party that is independent of the development organization (i.e. IV&V Provider),
(b) executes the V&V process that assures the products of ICT project meet the requirements and that the delivered system satisfies the intended use and user needs, and
(c) is proactive, whilst still maintaining its independence – its approach akin to auditing which is objective and impartial.
IV&V should be performed in a manner that provides early feedback to the development organization, where:
(a) it allows modifications to processes and products in a timely fashion,
(b) it results in fewer delays, reduced cost, higher product quality, and improvement of the development process itself.
By following the guidelines in this IV&V Handbook, The Agency can ensure that the IV&V Provider consistently delivers the day-to-day quality and performance requirements in a timely and cost-effective manner.
Engaging IV&V Provider therefore delivers:
(a) an objective system development appraisal,
(b) a baseline set of testable requirements that match the user’s needs,
(c) opportunity to identify problems early in the lifecycle,
(d) increased requirements and design visibility and traceability,
(e) early potential problem area indication,
(f) development risk reduction,
(g) improved maintainability and reliability.
The benefits of engaging IV&V Provider includes:
a) improving the overall ICT acquisition, where it focuses the project teams on eliminating defects earlier in the system development lifecycle that can lead to time and cost overrun,
a) enabling more sophisticated high-quality systems to be released to the public,
b) promotes the use of data-driven measurement and management methods for software engineering and analytical analyses of ICT project activities.
2.2.3 IV&V Provider Independence
With IV&V services, IV&V Providers should:
(a) have experience and knowledge about both verification and validation activities and measurements,
(b) provide periodic reports that verify and validate the product under test throughout the development lifecycle,
(c) find problems earlier in the lifecycle,
(d) perform independent assessment during the IV&V Engagement in line with engagement objectives.
As illustrated in Figure 2.5 below, the IV&V Provider must be technically, managerially, operationally, and financially independent from the Development Team and at all stages of the development lifecycle i.e. requirements, design, build and test, implementation, and maintenance stages.
Figure 2.5: IV&V Provider Independence
Due to its paramount importance, it must be consistently and repeatedly emphasised throughout the IV&V Engagement that the IV&V Provider must preserve its independence in the following areas:
(a) Managerial,
(b) Technical, and
(c) Financial.