Yes – The ROI Argument For Test Automation Can Be Won!
The ‘State of Quality 2016’ study by Wipro suggested that over the last two years there has been an increase of 85% in test automation initiatives.
The growing need for digital transformation and rise of methodologies such as Agile and DevOps too have been instrumental in greater test automation adoption. Software development today is all about speed, flexibility, increased efficiency, and productivity. To complement this, the testing methodology has to be equally efficient and flexible. Given that regression cycles continue to grow incrementally, from sprint to sprint, automated testing emerges as a critical part of optimizing quality and reducing testing fatigue. However, while automation is attractive, a lot of organizations feel that it is an expensive proposition and often wonder if automation initiatives deliver on the ROI metrics. This post is an attempt to address that question.
To begin with, we have to understand that the ROI of the automated testing initiatives depends a great deal on the level of test automation maturity. Automation testing is all about increasing the test coverage for automated execution, reusability, maintainability and consequently of increased productivity. Just how much you choose to automate impacts the “returns”. Now to the other side of the equation – the “investment”. To calculate the ROI of test automation initiatives, organizations must take into account the Total Cost of Ownership (TCO) and use it as the denominator. Some of the things that Total Cost of Ownership must include are:
- License costs of the automation testing tools
- Test automation framework development cost
- Cost of hiring skilled people for developing the test automation
- Cost of training the people for automating the testing
- The effort of carrying out the automated testing
- The costs of maintaining the test automation
- Hardware and associated infrastructure costs associated with the creation of the test automation and for the environment required for carrying out the automated testing
Along with these points, you may have to factor in other resource costs, effort estimates, team member transition costs etc. to identify the actual investment made in the automation initiatives.
In order to strategically address automated testing, organizations may have to navigate certain challenges as well. These include:
- The time to automate
- The high maintenance requirements of test suites
- The lack of skilled testers who also know programming since automation demands programming knowledge
- The high cost of automation testing tools and the skills to support them
All these factors have dragged down the ROI of automated testing. However, given that automation testing is no longer optional, organizations need to look for an option that will help them achieve higher levels of test automation while using the resources at hand and with optimal utilization of time and money. To positively impact the ROI of automation initiatives it thus becomes imperative to reduce the overall cost of automation, reduce maintenance costs, better utilize the available resources, and increase the speed of automation.
Increasing the speed of automation is a challenge that deters a lot of automation initiatives. Given that we are in the age of changing requirements, automation teams have to be geared to execute additional test cycles within challenging timeframes. Traditional test automation initiatives often struggle to meet these requirements since the automation needs to be created by skilled programmers. On top of that, these programmers often may not possess the requisite functional knowledge. Hence a constant dialog between functional manual testers and automation engineers becomes necessary. That means you need time from both these people and any communication gap between the two can lead to an increase in automation rework. Here Scriptless test automation, using Qualitia, can be of immense help since this form of test automation does not demand programming or coding skills. Functional testers themselves can automate, thereby significantly reducing the communication gaps and rework.
Let’s take an example of automated test suite creation phase. This example is representative of many of Qualitia’s existing customers.
Assuming 2000 test cases are to be automated and the team uses 3 people to complete the project, here’s the difference between the cost of creation of automated test suite utilizing traditional automation techniques vs Qualitia’s scriptless approach.
|Project Data||Conventional Approach||Qualitia approach|
|No of Test Cases||2,000||2,000|
|Estimated Effort to create automation suite (person months)||12||8|
|Time Required for Framework Building (months)||2||–|
|Training time (months)||0.75||0.25|
|Knowledge Transfer for team member rotation (month)||0.50||0.25|
|Qualitia License Cost (Annual Subscription Licenses – US $)||4,500|
|Average Person Cost ($ monthly)|
|Cost Elements||Conventional Approach||Qualitia approach|
|Qualitia License Cost||–||13,500|
|Test Suite Development||222,000||168,000|
|Team Member Transition||3,750||1,500|
|Opportunity Cost for delayed go-live||Opp. Cost exists||NIL|
|Total Cost of Ownership||291,750||191,250|
|Absolute Cost Saved in Year 1||100,500|
|Minimum Cost Savings % (Without Opp. Cost)||35%|
Using Qualitia’s scriptless automation approach, the time to automate is reduced by almost 60% considering there is no need for special coding and programming and even manual testers can automate testing with ease.
In addition, Qualitia’s smart alerts, end-to-end traceability and scriptless debugging features help in reducing maintenance costs by almost 70%. The ease of use of the product also ensures greater productivity gains and helps QA teams focus on achieving the highest index of increased test coverage. All these factors impact the Total Cost of Ownership positively, especially in areas such as time and effort, people and maintenance costs.
This typical example shows how Qualitia driven test automation helps provide better test automation ROI by increasing the speed of automation and reducing the costs that go with it. Are you convinced yet?