Section 2.3 Guiding Principles of IV&V

To ensure the objectives of IV&V in the ICT project are achieved, several guiding principles based on the industry best practices and philosophies must be applied when engaging an IV&V Provider. There are ten (10) guiding principles of IV&V that:
(a) shall be applied when implementing IV&V and engaging the necessary IV&V Provider and resources,
(b) are necessary to be ensured and enforced on the relevant stakeholders, where applicable, and
(c) depending on the aim of the principle, it may apply to only one or many of the stakeholders.

The guiding principles are categorized as follows.
(a) Essential Principles which are defined as principles or rules that are absolutely necessary and/or extremely important to be adopted and applied in IV&V Engagements and implementation, and
(b) Suggested “Good-Practice” Principles which are defined as principles or rules that are highly recommended for application in IV&V engagements and implementation, depending on the nature and objectives of the IV&V engagement.

Figure 2.6 below summarizes the Essential Principles.

Figure 2.7 below summarizes the Suggested “Good-Practice” Principles.

 2.3.1  Essential Principles of IV&V

The Essential Principles i.e. Principle 1 to Principle 4, guides the process for hiring and engagement of IV&V Providers by The Agency. The following explains each of the Essential Principles in more detail.

Principle 1: IV&V must be independent at all stages of the systems development lifecycle (i.e., requirements, design, build and test, implementation, and maintenance)

One of the most important principles is concerning the IV&V Provider’s characteristics. Consistent with IEEE Std 1012-2012 “IEEE Standard for System and Software Verification and Validation”, IV&V Provider qualifications require Managerial, Technical and Financial independence both in reality and perceptions.

These can be achieved via the following:
(a) For managerial issues, the responsibility for testing effort must be with the appointed third-party IV&V Provider to ensure quality-driven activities take place throughout the ICT project lifecycle.
­ The Development Team(s) and the IV&V Provider must have separate management structures. The IV&V Provider brings expertise to oversee the testing plans and activities are performed to the level required.
­ IV&V Provider will not be responsible for operational systems development tasks as it might be considered a conflict of interest. The IV&V Provider is allowed to provide both QA and IV&V services, while remaining independent of the development activities.
(b) For technical independence, the third-party IV&V Provider must not provide development or integration services or fix defects on the system being tested.
(c) For financial independence, the testing budget must not reside in the Development Team, but the budget should reside at a higher level of The Agency’s organizational management to ensure sufficient resources are allocated and dedicated for the IV&V Provider’s testing process.

 

Principle 2: Agencies must include scope and specifications of IV&V services it requires in the Request for Proposals (RFPs) for software development and augmented testing (2 coordinated contracts)

This principle relates to the IV&V services listed in request for proposals (RFPs) and tender documents that are issued to IV&V Providers and tender bidders. 

(a) All development contracts must identify a distinctive independent provider of IV&V services. IV&V contracts should specify objectives for:
­ economies of scale,
­ enabling The Agency to focus on core services, and increased process efficiencies,
­ IV&V Provider to provide progress reporting and automated records management systems,
­ the IV&V Provider and The Agency to mutually determine lessons learned Post Engagement.

(b) The following information must be shared with IV&V Provider via the Request for Proposals (RFP) and/or tender document:
­ Project objectives, deliverables and descriptions.
­ The Agency’s environmental factors.
­ The Agency’s process assets (policies, procedures, guidelines, and governance) that are relevant to the IV&V Engagement.
­ Service scope (specific work; descriptions; and deliverables).
­ The minimum test level, test type and number of test iteration / cycle expected from IV&V Provider.

 

Principle 3: IV&V Engagement must not be left to the last stage 

This principle is related to when best to engage the IV&V Provider. The IV&V Provider’s work will also involve reviews of large quantity and wide scope of deliverables throughout the development lifecycle, and thus cannot be performed at the end of the ICT project lifecycle.

Ideally, IV&V Provider involvement should be from the start of the lifecycle or “early-testing” and continues throughout the development process.

IV&V Engagement contracts should be finalized at the same time as the software development contracts in order for the IV&V Provider to begin work immediately. The IV&V Provider facilitates the accountability and increased communication among the stakeholders needed to build quality into the product and to use in the development and testing process.

The IV&V provider should attend relevant meetings (such as project’s Technical Committee meeting) in the lifecycle once the project is resourced, in order to maintain enough project understanding to offer input at the Quality Gates established throughout the engagement.

 

Principle 4: As The Agency’s partner, IV&V Provider represents The Agency’s interests 

This principle highlights the role of the IV&V Provider as an “additional and expert resource” to The Agency.

Throughout the engagement, the IV&V Provider represents The Agency’s interests and quality goals. This involves:
(a) taking care of The Agency’s positions and needs with regards to quality and successful IV&V in the project,
(b) working with The Agency as a partner to represent The Agency’s interests for high-quality outcomes in ICT projects,
(c) deploy skills needed to prevent conflict, resolve disputes, reconcile differences, and resolve issues quickly,
(d) improve communications, prevent misunderstandings, and build relationships with mutual trust among stakeholders that may have competing interests in the project.

2.3.2  Suggested ‘Good Practice’ Principles of IV&V

The remaining six (6) principles below i.e. Principle 5 to Principle 10, are suggestions for adoption to enhance a successful IV&V Engagement.

 

Principle 5: Staff and skills augmentation may be key driver for IV&V Engagements 
One of the most important suggested principles concerns skills augmentation, where the responsibility is with the IV&V Provider to:
(a) demonstrate strong technological skills and competencies, e.g., through education backgrounds and certifications of its IV&V Team personnel,
(b) demonstrate strong soft skills in communication, interpersonal skills, managing people, leadership, etc. to build relationships across The Agency,
(c) share and/or demonstrate their knowledge and skills to The Agency’s workforce, with the goal of transferring knowledge.

 

Principle 6: IV&V Provider may provide services relevant to technical systems and standards being used

This principle highlights the IV&V Provider’s system expertise, where The Agency should select an IV&V Provider based on their specific capabilities to match the project’s IV&V skill requirements.


(a) For example, if SAP is being implemented, IV&V Provider will need to provide resources with SAP expertise.
(b) The IV&V Provider must have similar experience in the system-type being implemented.
(c) The IV&V Provider must know Institute of Electrical and Electronics Engineers (IEEE) Standard (STD) 1012-2012 “IEEE Standard for System and Software Verification and Validation”, and other relevant standards including:
    - ISO / IEC / IEEE 29148:2011 for Requirement Engineering, and
    - ISO / IEC / IEEE 29119:2013 for Software Testing.
(d) IV&V Provider qualifications must include prior experience in similar engagement, context, technology, functional area, etc.
(e) Given the growing complexities of highly-connected systems, IV&V Provider should have a certain depth of similar experience to be effective. Also helpful is to require the IV&V Provider to have a background of IV&V works with past clients in:
    - similar industry,
    - same technologies being implemented (i.e., SAP or Java coding, etc.),
    - same functional areas (i.e., accounting, security, or web interfaces).

 

Principle 7: IV&V Provider may be asked to verify design documents mapping to requirements before coding starts

(a) This principle discusses ICT project lifecycle principles related to engaging an IV&V Provider.

(b) At each Quality Gate established by The Agency and agreed to by the IV&V Provider, if required, the IV&V Provider may verify:
    - requirements documents - by reviewing its testability, quality characteristics etc.,
    - functional aspects of the system, including usability – by reviewing their trace to functional test specifications etc.,
    - design documents - by matching them with requirements derived from user interviews, focus workgroups,                      document  walk-throughs etc.,
    - design specifications before coding – by reviewing their trace to system test specifications etc..

(c) The IV&V Provider may also evaluate design alternatives and validates for logical process/data models traceability to:
    - physical models,
    - the architecture,
    - hardware/software selections,
    - inputs/outputs,
    - data storage,
    - programs, and
    - initial system specifications.

(d) The IV&V Provider will play a strong role in maintaining a quality-oriented focus by monitoring for design mistakes, such as:
    - reducing design time,
    - feature creep,
    - silver bullet syndrome,

 

Principle 8: IV&V Provider may facilitate approval for system documents and user documents 

This principle discusses how the IV&V Provider performs document reviews. At each established Quality Gate, The Agency may request the IV&V Provider to conduct document reviews so as to provide timely feedback. The review feedback can be the basis for The Agency’s approval of technical documents.

Thus, IV&V Provider’s role in assisting The Agency in approving documents, may include:
(a) Ensure that projects are being executed according to international standards, project management, and software engineering processes, best practices & lessons learned.
(b) Ensure requirements are being met satisfactorily.
(c) Identify and ensure risks and challenges are properly addressed.
(d) Evaluate descriptions of all the possible responses (i.e., how the system is proposed to react) to an event that triggers the system.
(e) Evaluate configuration management processes and activities.
(f) Examine functional and non-functional requirements.

 

Principle 9: IV&V Provider may be involved in Project Management Office (PMO-type) operations

This principle discusses how the IV&V Provider should address project management.

An IV&V Provider will play an active role in PMO decisions with respect to requirement designs and implementation testing. Significant project management decisions will have IV&V Provider’s input, especially when related to issues of ICT acquisition quality.

The IV&V Provider must assess the PMO’s ability to be a strategic driver for organizational excellence related to quality. The IV&V Provider may value add by providing support to the PMO as follows:
(a) enhance project execution – example: by reviewing and enhancing project risk mitigation, by ensuring the IPP is well coordinated etc.
(b) project governance – example: by establishing project policies and charters, by reviewing authority levels etc.
(c) change management.- example : by identifying piloting scenarios, formulating a Change Implementation Plan after pilot etc.

 

Principle 10: IV&V Provider may provide risk management reports and monitoring services

This principle discusses how the IV&V Provider performs risk management. The goal of risk management is for risks and issues to be identified as early as possible in the lifecycle, and this can also be a major scope of work for the IV&V Provider services.

Examples of risk management activities by IV&V Provider include the following:
(a) Ensuring an objective view of ICT acquisition process execution and advising The Agency.
(b) While development vendors can have competing reasons for sharing issues, sometimes a client is not equipped to raise pertinent issues, and the IV&V Provider can assist in this area.
(c) Given that software test teams will not be able to conduct exhaustive testing, the IV&V Provider will ensure management of what to test and what not to test through risk-driven testing.
(d) Assist in finding as many defects as possible to ensure cost effective defect fixing.