Blog

In this article, we will cover the two basic types of testing: manual and automated. We will also talk about the way of testing in EuroLinux.

Software is an integral part of the modern world. We use it in all areas of our lives (e.g. smartphones, computers, laptops, tablets, cars). Malfunctioning software can cause many problems, which can eventually lead to loss of money, time or trust in the company. To reduce the risk of malfunctions when using software, or to assess its quality, it is necessary to subject it to tests.

What is software testing?

It is a common misconception that software testing is about running tests, by running them and checking the results obtained. Nothing could be further from the truth. Proper testing is a process that involves activities beyond just performing tests. It includes test planning and supervision, test analysis and design, test implementation and execution, evaluation of completion criteria, reporting and test closure activities.

Manual testing

Software can be tested either manually or by using appropriate tools. Manual testing of software, due to the fact that it does not require sophisticated tools, is more versatile. It is performed by a tester. It is them who checks all the relevant features of a given software, prepares test scenarios and generates reports. This is hard and time-consuming work that, costs a lot of effort, requires focus and is subject to the risk of human error. To optimize this way of testing, automation is being introduced.

Automated testing

Automated testing involves preparing test scripts or programs to automatically execute tests. Automated testing is based on pre-programmed tests that run automatically and compare actual results with expected results. A tester who verifies the execution status of the scripts is able to determine whether or not the software works as expected.

The main benefits of automating the testing process are speed, repeatability, efficiency, relevance and accuracy. We reduce repeatedly doing the same work, eliminate the risk of human error, increase uniformity and repeatability, objective evaluation and easy access to tests. However, introducing automation requires an investment of time and money. Time is needed to select the right tool, learn about it, and create, correct and execute scripts. It also takes money to purchase or create the appropriate tools.

How to test – manually or automatically?

Despite the many advantages of automation, manual testing is still essential. The key is to use them where automation won’t give us as accurate feedback. They are used to create exploratory tests or perform more creative test cases. This makes it possible to find many defects that might have been overlooked using automation alone. On the other hand, with more complex software, you may find that some tests can be much more efficiently executed and verified manually. Especially in areas where it would take a relatively long time to automate a particular process.

The process of testing in EuroLinux

In practice, manual testing should always be done before automatic testing. Therefore, at EuroLinux, we first perform manual testing of our software. In this way, we manually test its features, functionalities and compliance with documentation. Then our testers create automated tests based on them, using openQA and Jenkins. OpenQA is an open-source tool that allows a holistic analysis of the entire installation process of the software we produce, as well as visual testing of individual applications. The tests are written in the Perl language. Jenkins is also an open source tool for automation related to software development. With the help of our own script, it has been integrated with openQA.

Below are sample screenshots of the EuroLinux test process in OpenQA:

EuroLinux 9 ISO installation image test interface with a list and status of the modules included in the test.

EuroLinux 9.1 ISO image test results.

Checking specific items on the virtual machine’s screen by comparing them with a so-called “Needle” that specifies the items to be searched for and a list of tags to decide which needles will be used at any given time.

You can read more about the use of OpenQA in EuroLinux in the article openQA – or how we test EuroLinux on our blog.

Summary

In the increasingly fast-paced world of software, it is without a doubt that test automation is the future and a necessity. It reduces the time it takes to build and deploy software. In the long run, it saves time, money, reduces the error prone nature of testing, and allows more efficient use of testers. It is an added value to the organization that brings meaningful, positive business results.