Artifacts Used In Software Testing
Updated: Sep 18, 2021
1. Test Policy
Test Policy is the document that describes the testing process to be adopted at the organization level. It is prepared by the senior management and is used to streamline the testing process throughout the organization. It describes things like “What does testing mean for the organization?”, “How to improve the effectiveness of the testing process followed in the organization?” and so on. Generally test policy is used in organizations which have the desire to follow common testing process throughout the organization. Contents of the test policy are:
a. Advantages of testing for the organization which justifies the cost to be incurred in following the test processes.
b. Methods to measure test efficiency and effectiveness
c. Ways to enhance the testing process throughout the organization
2. Test Strategy
Test Strategy describes the test process to be followed at the department / program level i.e. the test process described in the test strategy document is followed by multiple projects handled by the department. It is generally prepared by the Test Manager handling multiple projects associated with the department. It describes the following items:
a. Objective of testing
b. In and out of scope items
c. Test Types (e.g. Functional and Non-functional)
d. Test levels (e.g. Unit, Functional, Integration, and System)
e. Entry and exit criteria for different test levels
f. Risks to be addressed
g. Test environment
h. Test case design methodology
i. Tools to be used (for automation testing and bug reporting etc.)
The test process defined in test strategy should be aligned with the organization’s test policy.
3. Test Plan
Test Plan is prepared at the project level and it is generally prepared by the test lead of the project. The test process defined in the test plan is specific to one project and has nothing to do with the test processes getting followed in other projects in the same organization. Items defined in the test plan could be similar to the items defined in the test strategy, the only difference is that the items defined in the test plan are specific to just one project whereas the items defined in the test strategy are associated with multiple projects. The following items could be described in a test plan:
a. Objective of testing for the project
b. In and out of scope items for the project
c. Testing schedule
d. Roles and responsibilities of each member of the test team in the project
e. Test case pass/fail criteria
f. Test deliverables etc.
The test process defined in test plan should be aligned with the organization’s test policy and the department's test strategy.
4. Test Scenario Document
Test Scenarios are prepared from the System Requirement Specifications (SRS) documentation of the project which contains the requirements to be implemented in the project. It is the role of the Business Analysts in the organization to collect the business requirements from the client. Once the business analysts collect the requirements they prepare the documentation for the same. Testing team then go through that document and prepare high level test scenarios from the same which could be further broken down into low level test cases. Test scenario document basically contains those requirements for which testing needs to be performed and it is prepared by the testers. Test scenarios are also called test conditions. Test Scenario should contain the following items:
a. Test Scenario Id
b. Test Scenario description
c. Name of the requirements document that was followed in preparing the test scenario
d. Serial number of the requirements document that was followed in preparing the test scenario
e. Section of the requirements document specific to the test scenario
Preparing test scenario document is optional hence it is not implemented in some of the projects. It all depends on the test process getting followed at the organization, department and project levels which describes whether the test scenario document will be prepared or not.
5. Test Case Document
Test cases are prepared in order to fully test that all the requirements of the software are implemented properly. Test cases could be written from a test scenario document or directly from the system requirements specifications (SRS) document. Testers are the people responsible for preparing test case document. Test cases document should contain positive as well as negative test cases. Positive test cases means that the outcome of the test case is positive, for example person who enters correct username and correct password should be able to successfully log into the application. And negative test case means that the outcome of the test case is negative, for example person who enters incorrect login credentials should not be allowed to log into the application. Following items are used in defining a test case:
a. Test case Id
b. Test case objective
c. Steps to be followed
d. Expected result
e. Actual result
f. Test case passed or failed
Test case document should be prepared in alignment with the organization’s test case design methodology (If any).
6. Bug Report
Bug report contains all the bugs that the software contains. These bugs are generally caught by executing the test cases defined in the test case document. All the test cases that are failed could be the bugs that the software has. A test case could fail because of many reasons, some of the reasons could be that the developer has forgotten to implement the feature or the feature is implemented in an incorrect way or the feature has been implemented correctly as per the requirement but there are some coding issues due to which the tester is unable to get the actual result same as the expected result while executing the test case. Bug reporting is a great way to identify the number of bugs the software has, also it is possible to segregate the bugs according to their priority, severity, status like open, closed etc. Bug report should contain:
a. Bug Id
b. Test case number
c. Steps to reproduce the bug
d. Priority of the bug
e. Severity of the bug
f. Screenshot of the bug or video footage of the steps followed (if required)
g. Any additional information relevant to the bug
Bugs are reported as per the organization level, department level and project level bug reporting strategy. In some projects bugs are reported in Microsoft word and Excel documents whereas in other projects bugs are reported in a bug reporting tool such as JIRA, BUGZILLA etc. and from there a bug report is prepared.