In our 1st gen of test automation efforts started as an “n-release” and Objective of Test automation to created Test data and reduced the manual effort for Regression testing, and Smoke testing. The automation script has been executed in the independent test environment and mostly commercial testing tools were recommended and used for script development.
Like Winrunner/QTP/UFT, IBM Rational, SilkTest and more …
Following challenges in the traditional test automation model.
- Manual test cases are automated and 20% to 35% functionality were covered through automation script and most of the time test automation is a kind of checking not an actual testing,
- If the test scripts were failed, automation tester will check with the manual tester and revalidate that functionality again.
- No matter how much we have been tested through automated script, there’s always functionality needs to be tested through the manual tester.
- Automation tester always depends on manual tester inputs. such as test cases, test data and defect retesting. Automation tester does not have a domain knowledge since they are involved in the later stage in the SDLC.
- Quality measured by Manual testing and around 70+% of test cases were executed by the manual tester in the each release and more rework/duplicate on both sides.
- In the traditional testing model following defect(s) taking more time to investigate, such as Environment issue, Interface, Program /functional defects, most of the time defect(s) are fixed in the separate branch to branch and development and testing efforts are higher.
My analysis 60+% of defects are opened during the manual testing. In automation standpoint, mostly functional defects are identified and program defect(s) raised in the later stage and fixed.
Having a strong domain knowledge is an added advantage for manual testing. However, our market expectation has changed to tools and technology. In our testing market, open source tools are available and Zero Cost and start-up companies promote their software testing tools with reasonable cost.
In current trends of testing, the Testing team has moved to DevOps-QA Model, “DevOps QA nothing but continues Integration testing”. The maximum no of test cases are executed on multiple time in the shared environment, So 50% to 60% of test cases can be covered on UI Automation and API level testing in the short span of time or within the sprint level. Cross browser testing in the multiple platforms and parallel executions are more effective way to test the functionality and identify the bugs as much early.
Continues Integration testing approach is a mindset, CI Tools major player for current automation, my favorite tool Jenkins is an extensible continuous integration tool, DevOps engineers(Dev and Automation Tester) have an easier time integrating changes to projects and have access to outputs to easily notice when something goes failed. The entire code base is integrated build and automated and tested constantly, the potential bugs are caught earlier in the life cycle which test results are transparency in the better quality and quick delivery to market.
Jenkins’ Key Features:
- Distributed builds & Notification.
- Integration with third party plugins.
- JUnit/TestNG supports and reporting.
- Scheduled and easy tracking.
- Customized Dashboards.
- Support version control.
- Cross browser Test reports and more . . .
DevOps-QA effort will vary based on Industries and reason behind was business, domain, legacy application, Project timeline, budget, resources skills, infrastructure and more…
Following factors are more important for DevOps-QA ( Continues Integration testing).
- Tester First:
Don’t fill the headcount. Find the right people or Provide the training. The tester must participate the project discussion from day 1. this will help to understand the domain and testing have to happen in parallel mode.
- Technology: Complex web, mobile web, and mobile Native or hybrid applications is a right choice to start DevOps-QA (CI-Testing). if your company/business does not support the open source tools, choose the commercial tool(s) as per the company process.
- Test Infrastructure: This includes the organization’s policies, procedures, culture, test processes, testing tools and resources and most of the time it was compromised by test leaders due to cost and project timeline.
- Test data and test cases availability: Test Automation depends on Test data and detailed level test cases In test automation 30% of efforts has been depending on Test data identifications and creations and extensive test data is an entry point for test automation.
- Process: This is a culture change, continues improvement and learning. A series of process steps taken in order to achieve the customer demands. it’s all about a quick and effective way to market
Benefits of DevOps-QA (Continues Integration testing).
- CI-Test Automation becomes a critical part of business in the rapid changes in software.
- More test automation coverage and find the bug as much early.
- Reduce the manual effort both development and testing area
- increase the productivity and more robust model.
- Save the development effort and testing cost, High quality, and products always deliver on time.
- Meet our market demands and get the feedback.