Pros & Cons of going in for Automated Testing

When to use the Automation tools:
Great effort is required in firstly automating the test cases and maintaining them thereafter. Hence some sort of cost-benefit analysis is quite helpful before investing money and putting efforts into it.

The test engineer should evaluate whether the likely benefits of automation are fitting within the boundaries of the required improvement criteria and whether the pursuit of automated testing on the project is still logical & falling within the framework of organizational needs.

Automation is suited to following types of testing with specific intentions:
1) Functional Testing – on operations which perform as per the expectations.

2) Regression Testing – on the behavior of the system which has not been changed.

3) Exception Testing – thereby forcing error conditions in the system.

4) Stress Testing – to determine the absolute capacities of the application and operational infrastructure.

5) Performance Testing – to provide assurance that the performance of the system will be adequate for both batch runs and online transactions in relation to business projections and requirements.

6) Load Testing – to determine the points at which the capacity and performance of the system become degraded to the situation that hardware or software upgrades would be required.

Benefits of Automated Testing:

1) Reliable: Tests perform precisely the same operations each time they are run, thereby eliminating human error. Hence it produces a reliable system.

2) Repeatable: You can test how the software reacts under repeated execution of the same operations.

3) Programmable: You can program sophisticated tests that bring out hidden information from the application.

4) Comprehensive: You can build a suite of tests that covers every feature in your application.

5) Reusable: You can reuse tests on different versions of an application, even if the user interface changes.

6) Better Quality Software: It Improves the quality of the testing effort. Because you can run more tests in less time with fewer resources

7) Fast: It reduces the test effort by minimizing the schedule. Automated Tools run tests significantly faster than human users.

8) Economical: As the number of resources for regression test are reduced.

Choosing the right tools for the job and targeting the right areas of the organization to deploy them can only realize these benefits. The right areas where the automation fit must be chosen.

Disadvantages of Automation Testing:
Though the automation testing has many advantages, it has its own disadvantages too. Some of the disadvantages are:

# Proficiency is required to write the automation test scripts.

# Debugging the test script is major issue. If any error is present in the test script, sometimes it may lead to deadly consequences.

# Test maintenance is costly in case of playback methods. Even though a minor change occurs in the GUI, the test script has to be re-recorded or replaced by a new test script.

# Maintenance of test data files is difficult, if the test script tests more screens.

Ultimately how to decide – Whether to go in for Automated Testing or Manual Testing?

1) Based upon the frequency of use of Test Cases:
Automating a test case requires almost 3-4 times the effort as compared to manual execution of it once. To draw benefit out of significant investment in the automation tool, we should be able to execute the automation script the maximum number of times or at least 7-8 times. There is no worthwhile idea of going in for an automation tool for short-term product, which can be easily managed by manual means. For products involving many rounds of testing, the use of an automation tool is understandable.

2) Time comparison:
Automated versus Manual Testing: Generally an automation script runs much faster as compared to manual execution. However post execution activities are considerably time consuming. After running the automation script fully; time consuming activities are the analysis of test results & investigation or identification of actions causing failure at defined checkpoints. Whereas in case of manual execution of the test script, there is no need of any separate time for analysis of the results, since the actions causing failure at checkpoints become already known.

Hence automated testing is viable only if, the combined time spent on running the automation script as well as doing post automation analysis of test results is significantly less in comparison to the time spent on manual execution. However nicely developed automation scripts do not need constant monitoring and can run without any manual intervention. In such a case automation can be greatly productive & can cut down the time of running the scripts.

However when large number of regression issues are there after fixing the bugs, automation is the best alternative. Manual testing can be extremely time consuming in such cases.

3) Reusability of Automation Scripts:
The automated testing is viable in case automation scripts are reusable in future as well. One may be compelled to think again in favor of automation if considerable effort is expected to be made in upgrading the automation scripts. The return on investment made on the automation tool can be maximized by reusability of the automation scripts with small modifications.

4) Stability of the Product under Test:
Automation scripts are not advisable for use on software product, which itself is not adequately stable. Frequent changes in automation script are not desirable, thus unless the product acquires enough stability during its development cycle, there is no point in automation unless we are operating in an agile environment.

5) Adaptability of test cases for automation:
A statement that, "All test cases can easily be automated" – is not true. Many a times we land up with some type of test cases, which are not worth automating. Hence there is no point in wasting automation effort over such test cases. For complicated product with tightly integrated bunch of many applications, running the automation script again & again in the event of stoppage of the test case, can become a pain in the neck. In such cases tester would certainly prefer to run the script manually & save considerable amount of time as compared to running the automated script.

6) Exploitation of automation tool:
If we want to go in for automated testing of a product, we must draw full benefit out of the test tool to draw the maximum return on its investment. The automation tool should be deployed for performing less complicated / time consuming & repetitive tasks. This will help the test engineers in concentrating their time & energy on other significantly important tasks. There is no point in automating highly complex test cases, which can easily be executed manually.

Automation tool should be deployed to test the breadth of the application under test. However manual test engineers can handle the in-depth testing more efficiently.

Manual testing can not be totally eliminated in any way. Automation scripts have both advantage and disadvantages in the sense that they perform actions exactly in the way they are coded to perform without any deviation. For even a slight deviation, the script needs to be changed. Running the same automated script again & again will not detect more bugs. However to detect more & more bugs, we need to move across the side & have little deviation from the flow, & such actions can be better accomplished by manual testing.

When a test case is perfectly designed & made to execute from end to end automatically through the tool without, it automatically verifies all predefined check points without any manual intervention.

7) User simulation over large web application: For simulation of several virtual / dummy users interacting over a large web application, load testing automated tools like LoadRunner etc. can be easily deployed to establish the load bearing functionality of the application. Such load testing by manual means is extremely difficult. Thus in spite of many disadvantages & harms from automated scripts, the automation testing is adapted widely all over the world.

Key Word: Automation Testing, Manual Testing, Pros & Cons