What are the disadvantages of regression testing

QA dilemma: manual or automated testing

There has been longstanding debate among quality assurance (QA) specialists about the advantages and disadvantages of manual and automated testing and the difference between the two approaches. The reason for this is pretty logical: software testing is an important phase when developing an IT product. At Innowise we understand exactly why testing is really important. That is why we offer services for testing web apps and mobile apps on request.
We hope that after reading this article, you can decide which of the two options - "manual or automated testing" - is better for you. Let's talk about these two terms first.

What is the difference between automation and manual testing?

During manual testing (MT), a QA engineer or tester carries out each test case by hand, no scripts are used. With automated testing (AT) everything is reversed: a tester uses scripts, tools and software to run all test cases. Now that we understand both concepts, let's delve into details.

Automated testing

Test-Driven Development (TDD) is not possible without automatic tests. In TDD you first write unit tests and then the real code appears. Once the developer finishes the code, it is time to run unit tests (typically, unit tests are done by the developers). Then either this code is refactored or other tests are added and executed, the order of the subsequent actions depending on the result.

Here is the list of test types where automation is better than the manual approach:

Regression tests

Automated testing is perfect for regression testing because the developers change the code from time to time and the regressions can run promptly

Functional testing

The effective capacity of an app's functional specifications is determined when the software is tested. In general, compliance, compatibility, accuracy, security and relevance are taken into account

Testing the graphical user interface (GUI)

This includes UI testing to see if the GUI meets all of the required requirements

Repeated execution

If you want to perform a task repeatedly, this type of testing is a good choice

Performance tests

Similar to the previous case: if a large number of users are to be simulated at the same time, automated testing is the option.

When do I choose automated testing?

1. When the tasks you have take up a lot of your time.
After you have reached a stage when all the necessary changes and modifications have already been made, you need to verify that they are all working properly. Of course, you don't want to keep checking all of the app's functionality, as this is not only much better, but it also reduces the overhead of running regression tests. This will help you find the latest bugs and save time and effort.

2. When you have to deal with repetitive tasks.
There are cases when you keep adding new features that are very similar, or even identical, or you keep doing identical operations on your project. Leave all of your problems to automated testing and save time.

3. If no human error is to occur.
All automated tests are carried out with the help of special tools and scripts. Therefore, the risk of failure to be overlooked is minimal. Because of this, it saves time and is something to be relied on.

4. When complex GUI tests are expected.
AT not only identifies, but also logs ways in which platforms behave differently. Testing the graphical user interface is also useful when looking for regression errors.

5. When load and performance are to be tested.
In all honesty, there is no way to manually test load and performance. Thousands of simultaneous users are simulated during the AT.

Keep the following in mind: If you are certain that no features should change significantly during project development, automated testing is a good choice. However, if you expect functionality to change frequently, it is easier to test manually.

Manual testing

Now you are probably wondering why MT exists. Don't underestimate it, anyway, because MT should be run before moving on to AT. It's true that it takes a lot of effort, but there is no other way to ensure that automated testing is not impossible at all.

We have prepared a list of tests that cannot be automated or that are difficult to automate:

Usability tests

A QA engineer analyzes how simple, convenient and handy the software or product is for the end user. Human testing is an essential part of usability testing. Therefore, this type of test should be performed manually.

Exploratory testing

In this case, a tester's analytical and logical skills, knowledge, creativity, experience and intuition are a must. Typically, such tests are accompanied by poor specification documentation and you cannot afford to devote much time to running them. This scenario requires human skills and knowledge to run the tests.

Ad hoc tests

There is no "right" way to do this type of test. A tester does not create plans and the system functionality is tested haphazardly. The challenge in this case is to be as inventive and creative as possible and do your best to crash the system to find bugs.

When is manual testing a better choice?

1. If it is a short term project.
On the one hand, the focus of the AT is not to spend a lot of time and effort, on the other hand, creating and supporting automated tests is both energy and time consuming. If you want to create a small website for advertising, AT is hardly required.

2. When you need UI acceptance tests.
This is the most common case when MT is preferred because the user interface and its visual elements are to be tested. The reason for this is very simple: Automated tests cannot identify gestures, image colors, text markings, font sizes and links. Imagine you've created an app and there is a bug: a link is very small that the user cannot see it. A human will notice such an error within seconds, while an automated test will not detect that there is a problem here.

3. When your project is in the early stages of development.
In this phase manual testers can help to find the bottlenecks in the project. This reduces the time and effort required to troubleshoot later. Some consider automated testing to be too expensive and resource-intensive for this volatile phase of product development. Others say that automated testing should be present at every stage of development.

4. When testing cannot be automated.
There are cases when automation is not possible. Usually these are tests related to hardware. It is hardly possible to create scripts to test touchpads, screens and sensors. Another point is that there is data that you can only test by hand. Of course, it is possible to run an automated test to see if a printer is actually printing. However, the manual approach is probably better if you are concerned about the quality of a printed image.

Summary

After all the discussion, we can broadly say that there is no answer to the question of whether test automation is better than manual testing or vice versa. The selection must be made according to your project. The best approach is to combine manual and automated testing. That’s what we’re doing in Innowise.

One of our C-level managers who encourages our company to achieve new development goals. He focuses on the quality of the work done, ensures that work with clients is proactive and motivates the team.