Image CAPTCHA
Enter the characters shown in the image.

You are here

ROI of Automation Effort

Automation effort, as any effort in software production process must be analyzed according to business criteria, in many cases the ROI is the first one. Is it worth ? Well, there is no single answer to that, or single formula. What metrics and formulas can be used in calculating ROI of an automation effort ? The simplest formula can be:

Em = Time it takes to execute the cycle manually

Mm = the sum time of updating the manual testing documentation after each run, per cycle

N = number of cycles that need to be executed

Au = Time it takes to automate the cycle

Ea = time it takes to execute full cycle automatically

Ma = time it takes to modify the automatic framework on changes between cycles.

So, according to time factor, automation is suitable if: N * (Em+Mm) > Au + N * (Ea+Ma)

But still this formula doesn't give us a direct answer, weather to do automation or not. The next factor that should be tested is Cost. Let's take the regression task for example:

Pm = regression cost on creating and executing manual testing (include number of workers, working hours, working place and terms) Pmu = update regression cost of manual testing Pa = regression cost of automating the cycle (tools price, developer working hours and terms, automation station cost) Paex = regression cost of executing the cycle Pau = update regression system between cycles (developer hours, automation station cost ) So according to cost criteria, automation is worth if N * (Pm + Pmu) > Pa + N * (Paex + Pau)

Since much of the Automation frameworks is done, when lifting up the automation platform, the ROI is even getting better on the next releases. What about testing the automation ROI against finding defect criteria ? Well I believe that it will be possible, only if the testing strategy is to test it all. Meaning full 100% coverage, of the new and old features, and since most of the software companies are working on testing according to probability chances, Risk Based Testing, most of the added value of the Automation system is to verify that the old features were well protected and not damaged because of the new features. Regression testing, Acceptance testing and verification testing cannot be measured by finding defect criteria. More advantages that were not taken in the previous formula:

  1. When having Automation capabilities, you activate them much more often, making sure quality remains after every change.
  2. Faster feedback for the developers teams, sometimes means less defect and much more faster manufacturing process.
  3. Higher overall testing capabilities even means that developing process become more object oriented, more modular, making sure that every change doesn't reflect other existing features. Developing process gets much more improved and efficient.
  4. Manual test switch their focus only on new features and edge situation testing which will cause the product quality level to jump much higher.
  5. Manual testers will catch much more defects, QA work will be much more efficient.
  6. Return defects level will be much less, or defects caused during work to fix others will be much less, I call it Backlash work, and there is even Backlash work on second level and third.
  7. Human resource costs are dropping down, even if you retain your manual testing power, you will need it to be more professional on the new features, and less on the overall product structure, learning curve will be dropped and getting professional will be much more faster.

Summary You can start with the simple RIO formula, but then you have to take in consideration all the other criteria according to your product, service and targets. You will have to accumulate your product values and targets into numbers, for example: How much it is worth to you to be first in quality level ? How much it is worth to you to be first and fast into market ? How complicated is your product, can you keep the high quality with profession less resources.? Are you thinking about using Outsourcing for development or testing or both ? etc. The answers for these questions are changing from one manufacturing process to another, and have a huge influence on the ROI that you will have on your Automation platform. 

MANUFACTURERS Wallboard

Testing tool manufacturers world-wide list
10Levels ABID CONSULTING AccelQ Accord Software ActiMind AdaCore
AdaLog AgileLoad AgileWay Agitar Algorismi ALL4TEC
Andreas Kleffel Android Apache Apica Apollo Systems AppAssist.mobi
Applitools AppPerfect Appsee ApTest Assertible Assure
Atlassian AutoIt Consulti .. Automation Anyw .. Automation Cons .. Axosoft Aztaz Software
Backtrace I/O Badboy BlazeMeter Borvid BrowserStack BSQUARE
BStriker Intern .. CA Technologies Canonical Canoo Engineeri .. Catch Software CelestialTeapot
Chris Mallett Cleanscape ClicTest CloudQA Codeborne CodeCentrix
CodePlex projec .. Codoid Cogitek Compuware Configure IT Conflair
ConSol Core Services Coronys Ltd Countersoft CresTech Softwa .. CrossBrowserTes ..
Crosscheck Netw .. Crowdsourced Te .. Cucumber Ltd Cyara Cygnet Infotech DareBoost
Databene Datamatics Glob .. DevExpress DTM soft Dynatrace LLC EasyQA
Eclipse EkaTechserv Elvior Emmanuel Jorge Empirix EPAM Systems
Equafy Esterel Technol .. eXept Software .. Experitest Finaris Froglogic
FrontEndART Ltd GeneXus GitHub project gnoso Google Code Pro .. GrammaTech
Gurock Software HelpSystems HENIX Hewlett Packard .. Hexawise High-Tech Bridg ..
Hiptest Hitex IBM Rational imbus Shanghai Impetus Inflectra
informUp InTENSO - IT Ex .. Ipswitch Jamo Solutions Janova JAR Technologie ..
JBoss Developer jClarity Jellly.io JetBrains Jively jQuery foundati ..
JS Foundation Jspresso Kanoah KMS Technology Kualitee LDRA Limited
Litmus LoadFocus Loadster Perfor .. MarathonITE Marketcircle Marketcircle
Maveryx Meliora Ltd Micro Focus Sof .. Microsoft Mobile Labs Mobile1st
Mockaroo, LLC Monkop Mozila MSys Technologi .. Navicat NeoTys
Neowise Softwar .. NetCart NORIZZK.COM Novosync Mobili .. NRG Global NTT Resonant
OC Systems Odin Technology OpCord Oracle Orcanos Original Softwa ..
OW2 PANAYA Parasoft PassMark Patterson Consu .. Perfecto Mobile
Pivotal, Inc. Plutora Postman (API To .. PractiTest PrimaTest Process One
Programming Res .. Psoda PureLoad PushToTest Python Q-Assurance
QA Systems QACube QASymphony QAWorks QMetry Quali
Qualitia Softwa .. Quality First S .. Quotium RadView Softwar .. Ranorex RedLine13
Reflective Solu .. ReQtest RevDeBug Robotium Tech Rogue Wave Soft .. Rommana Softwar ..
RTTS ruby-doc.org Runscope Sandklef GNU La .. Sauce Labs Seapine Softwar ..
SeleniumHQ Sencha Sensiple Siemens PLM Sof .. SmartBear Softw .. SmarteSoft
SOASTA SoftLogica Softomotive Softsmith Solution-Soft SonarSource
Sourceforge Spirent Technol .. SQS Software Qu .. Square Stimulus Techno .. Swifting AB
Synopsys T-komp T-Plan TechExcel TechTalk Telerik By Prog ..
Tellurium Test Collab Test Goat Test Recon TestCaseLab testCloud.de Gm ..
TestCraft Techn .. Testenium Testim.io TestingBot TestLodge Testmunk
Testomato TestOptimal TestPlant TestPro Testsigma Techn .. Testuff
The Core Bankin .. The MathWorks The Open Group thePHP.cc Thoughtbot Thoughtworks
Tigris.org Time Simulator Top-Q Trace Technolog .. TrendIC TRICENTIS
Tritusa Pty Ltd TWD Solutions P .. TypeMock Tyto Software Ubertesters UniTESK
Universal Test .. Usetrace Ltd Utrecht Univers .. Validata Group Vanamco AG Vector Software
Veracode Verifaya Corpor .. Verit VersionOne Viewlike.us Vornex Inc.
Watir.com WcfStorm Soluti .. We Are Mammoth Web Performance .. Wintask Wireshark Found ..
Worksoft Xceptance XK72 Xpand IT XQual ZAPTEST
Zeenyx Software .. Zephyr Zeta Software zutubi pty

Theme by Danetsoft and Danang Probo Sayekti