Key Information in a Test Plan

Test Plan is a document providing structural approach describing how testing will be accomplished in the Project. This acts as a contract between testing team and stakeholders on how testing is planned for the project. 

For successful execution of testing in the project, it is very important to do thorough planning of the testing requirement and documenting the same in the form of test plan.

Most of the organisation have defined template for test plan and have slight difference in the structure of the test Plan from each other. What information needs to be captured in the test plan will be discussed in this article in the first section:

• Defining the Scope of testing – The first and one of the important component in test planning is defining the scope of testing and at the same time providing the items which are Out of scope of testing. Scope of testing should be in line with the overall scope of Project. Defining scope early in testing helps in defining the acceptance criteria and agrees on the exit criteria for testing.

• Objective of Testing in the Project should be defined - Objective of testing can be testing the functional requirement or creating an automation suite for regression scope. Also there can be multiple objective of testing with some critical and some secondary objective e.g. : Testing complete functional requirement of application can be primary objective of testing with communicating all known defects and issue with different stakeholder and closure on all of them before release to production can be the secondary objective of testing. All the objectives should be clearly defined in test plan. Each objective should have the priority defined and acceptance criteria should be associated with each test objective.

• Staffing and training needs for the testing team should be mentioned in the test plan.

• Environment and resource requirement required for testing, i.e. Software and hardware requirements should be clearly defined.

• Assumptions acts as probable risks to Project. All the assumptions should be added in test plan and proper sign-off  should be taken from stakeholders to keep them informed and confirmation should be taken from them for the correctness of assumptions made during testing and will not lead to possible risks in future.

• We should define constraints in the test plan. Constraints impact the efficiency and effectiveness of testing. Constraint can include for e.g. staffing, time or cost constraints

To summarize, so far we have discussed the ‘What’ part of testing, i.e. What we will test, what will be our scope, what our assumption, constraints are, what will be the scope of testing. Once we are clear on What, Next question is how to do what.

How Testing will be executed is explained in Testing Methodology. The Key points explained in Testing Methodology include:

• Define testing strategies for different phases and functional and non-functional requirements. This includes:

• Different types of testing required for requirement.

• Approach for each testing type, e.g. Regression, functional, automation or locale testing.

• Risk/issues identification and test data generation for testing types.

• Testing cycle for each testing and testing milestones are set.

• Entry and exit criteria for each testing milestones are described

• Providing defect lifecycle to be followed.
• Defining testing schedule in sync with the milestone defined.

Another Topic to be covered in test plan includes test deliverables and defining the exit criteria for testing completion. Test deliverables should be defined; it can include user guide, test scripts, test data, Test execution results, and Test plan.

Together, we can add matrix for different areas in test plan for easier understanding and better explanation. We will discuss on methodology and test deliverables in details in the next articles on topic Software Testing.

Risk Analysis - Next Steps for Identified Risks

Once by using different Risk Identification techniques as described in previous article, we have identified all the possible risk in the project life cycle, Please see  First Step in Risk Analysis – Techniques for Identifying Risks,  The next phases in Risk Analysis post risk Identification phase in order of their occurrence  are as follows:

•  Use Organisation level risk Management Template or use Risk breakdown sheet. – Most of the organisation has templates defined for Risk Analysis. It makes no sense to reinvent the wheel. Some of key data that we need to store for each risk are defined as below:

a.    Risk Definition – This explains what the risk is.

b.   Root Cause – We need to analyze what is the root cause of the risk. Analyzing risk root cause help to identify where problem and it is highly possible that more than one risk have the same root cause. So countering the root cause using control can help solve multiple goals in one go.

c.   Risk Categorization – Categorization risk is important to group risk. Risk can be classified broadly into three categories, Technical, Internal or External risk. Classifying Risk into categories helps to get proper input from the experts or specific team. For this risks can also be sub-categorized. E.g.: Risk can be related to Infrastructure team or Legal team, So we get input and control actions from the expert group in one go, and helps to get proper inputs from the required team or individual.

d.   Once we have root cause, categorization, and risk definition. We need to provide the control action on how to avoid or reduce the impact of risk.

e.   When we define risk, the probability of happening of a risk can vary from project to project. Probability of risk occurrence needs to be understood and provide in the template

f.    Similar to Probability, Impact of risk is very important. Even a low probability risk can be highly critical in case its business impact is very high. Control action is must, and should be well thought of as it can affect business

.

g.  We also need to identify in which phase  risk can turn into a loss. E.g. : Assumption which are also risk should be resolved in requirement Phase and so on. Response of risk events occurring early in the project should be handled first and detailed risk mitigation plan developed for the same.

h.   Risk should be quantitatively analyzed especially with high impact, high probability, or both to generate statically cleaner data for risk occurrence and its impact. Decision tree, distribution models, and tornado diagrams are some example of the same.

i.   Define the expected monetary value of the risk converting to loss taking into consideration both impact and probability

j.    Next Step is to assign risk to required stakeholders. We can break the document into high to medium priority risk and low and trivial priority risks in another document. So that major focus is not diverted on low and trivial priority risks.

k.   Once we get the response, we need to maintain the template document used with the responses and response provider to clarify with the Risk Owner.

• Once we have created the template, we still require regular risk meeting and risk monitoring in process, so as to add any new risk identified during the project life cycle and also to track how we are faring with the risk identified

First Step in Risk Analysis – Techniques for Identifying Risks

The first phase in risk Analysis is identifying risks and categorization of risks. There are three categories of risks associated with software systems, Internal Risks, External Risks and Technical risks. Risk can be broadly classified into these categories. Categorization of risk into categories is known as Risk breakdown structure (RBS).

In this topic, we will discuss mainly on different risk identification techniques. In the next articles, we will discuss on what are the different types of risks.

Risk Identification Techniques:

The first step for a successful risk analysis is to identify all risks in the system during the planning stage. It is important for whole group to provide inputs in risk as individuals on specific stream are in better position to provide on the possible risks and collating all information in a RBS or specific template as used by Organisation for risk analysis. Every possible risk should be identified during the initial phase irrespective of its impact. Risk can be collected from team in following ways:

• Brainstorming – In this process, the group comprising of team members collects in a room and discusses on every possible risk in the presence of a facilitator who knows the risk processes. Users can discuss on different approaches, interviewing team members of group, including as much stakeholders as possible, discussing on lesson learnt and risks found in similar project in the past projects.

•  Delphi Technique – In this process, same questionnaire is sent to multiple experts by the facilitator asking for their opinions on the probable risks in the project. Once inputs are received from the expert, the facilitator shares the information back to the experts asking for their opinions again on the risks. Facilitator shares the rationale provided by different experts to the same risk with the same group. The process is continued for 2-3 rounds, and usually experts come on similar page provided based on collective rationale provided by team. For success of Delphi technique, anonymity of experts is a must and should be kept even after risk identification process completion.

• Root Cause Analysis – Once we have the identified risk, we need to know what is the condition that will cause the risk to happen. This also provides the insight on condition that may trigger a risk and it is very much possible that root cause of multiple risks be one. Using fishbone or Ishikawa technique can be used to generate risk information and extract useful information diagrammatically from the risk.

• Interview the stakeholder – Together with internal team, client can also be interview what are the different risk they feel on the project. Interviewing different stakeholders bring different perspective on the same issue.

•  Assumptions– For improper information in the project, we do make assumptions. Each of the assumption you make is a risk to the Project and we should have sign off as early as possible on the assumption we have made from client Perspective.

• Checklist Analysis – Analysis checklist of risk and discussing on the risk in checklist if the risk has been discussed, and a decision has been made. This ensures each of the risk in the system is analyzed, discussed and work upon.

• Lesson Learnt and documentation from past project – Risk identified should be documented in a document as previous documents and lesson learnt acts as an important sources of risk applicable in current work.

• Using SWOT – Creating a SWOT helps to identify the strength, weakness, Opportunities and Threats. Understanding Weaknesses and threats are an important source of risk identification

Understanding Risk Analysis flow

Risk is the potential loss to an organization. Projects usually have weaknesses or environment that may present threat to the project. The weaknesses can be called as vulnerabilities. Risk identified during planning phase needs to have some corrective actions called as controls which will come into action, if some risk is triggered. The controls are expected to reduce the risk or remove the risk as much as possible. Although controls and risk analysis can reduce the risk, but we can act as much as possible to reduce risk using the process.

 The risk is turned into a loss by threat. A threat is the trigger that causes the risk to become a loss. While it is difficult to deal with risks, one can deal very specifically with threats. Threats are reduced or eliminated by controls. Thus, control can be identified as anything that tends to cause the reduction of risk. If our controls are inadequate to reduce the risk, we have vulnerability. Vulnerability, therefore, can be defined as a flaw in the system of control that will enable a threat to be exploited. Risk Analysis is the process of evaluating risks, threats, controls, and vulnerabilities 

Scenarios that can happen based on how risk was handled during planning.

Case 1: Risk identified in Risk Analysis and proper control identified in case such scenario arise in future. Loss will be minimized or risk will be avoided in this case. Even in case of loss to Organisation, Stakeholder would be satisfied as the risk was expected to happen and with control in Place, the impact of the loss was reduced considerably

Case 2: Risk was identified during planning phase but proper control were not provided during analysis, thus loss was reduced but still the scenario could have been done in a better manner.

Case 3: In case no risk Analysis is in Place, in case a threat is triggered, It sends out panic within the team, though some action will be taken by stakeholder, but chances of loss to organisation are very high and it is highly possible that it is too late to correct the mistake and the loss has already done to the project. Thus impacting team, confidence with client, client business and relationship and own business also.

Once a risk happen, we have to take either of the control action which are described below:

• Avoid the Risk – We should try to avoid the risk to convert into planning control activities to avoid risk.

• Mitigate the Risk – In case a risk becomes an event and control were defined during planning to reduce the risk. We should have mitigation controls in place already defined during planning that would reduce impact of the risk.

• Transfer the Risk – Insuring the project or transfer of work to specialized external team is a good way to transfer the risk in case avoiding the risk or mitigating the risk are not feasible .

• Accept the Risk – There may be risk that does not have any solution and are not able to be channelized using either of above control action. An example could be change in regulations for a regulatory work. Such risk although should be analyzed during risk Analysis Phase

Risk Analysis should be done at different levels, and not only at leadership skill. Even risk Analysis should be done in everyday life periodically to identify if we are on right track or not.

In this series, in next article we will discuss on Risk templates, common risks in Quality, types of risk and how risk analysis happens