Completed Cortex Academy Meetup on the Topic of Software Testing
Last night, another meetup in the Cortex Academy meetup series was held, primarily aimed at Cortex Academy students but open to anyone interested in getting direct answers from top experts and resolving doubts about ICT professions.
The theme of the 4th Cortex Academy meetup was software testing, and the moderator, Jovan Kovačević (CTO of Coinis), had a conversation with Marko Dragović (Testing Expert at Amdocs) and Branko Bukilić (CEO of Data Design).
If you couldn’t attend the meetup, you can watch the recording on our YouTube channel: YouTube Link
This meetup was marked by significant interest in quality professional discussions and numerous questions. Due to time constraints, we couldn’t answer all the questions during the live session, but our guests have provided written responses, which you can find in the following blog post:
QA Meetup – Answers to Submitted Questions
- Is Selenium or Cypress better? Is Playwright better than Selenium? How do you automate mobile applications? Marko: You can find a comparative table for all three tools (Playwright, Cypress, Selenium) below.
- Appium is a popular open-source tool used for the automated testing of mobile applications. It allows developers to automate testing of native or hybrid iOS and Android apps. Appium doesn’t work alone; it runs test cases using the WebDriver interface. Similar to Selenium, Appium enables testers to create test scripts in multiple programming languages such as Java, JavaScript, PHP, Ruby, Python, and C#.
- Appium is particularly favored because it is a flexible, cross-platform framework that testers can use to create test scripts applicable to multiple platforms (Windows, iOS, and Android) using the same API. In essence, Appium users can reuse their code for both Android and iOS, reducing time and effort.
| Criteria | Playwright | Cypress | Selenium |
| Language | Supports multiple languages such as JavaScript, Java, Python, and .NET C# | Supports JavaScript | Supports multiple languages such as Java, Python, C#, Ruby, Perl, PHP, and JavaScript |
| Test Runner Frameworks Supported | Mocha, Jest, Jasmine | Mocha | Mocha, Jest, Jasmine, Protractor, and WebDriverIO |
| Operating Systems Supported | Windows, Linux, and macOS | Windows, Linux, and macOS 10.9 and above | Windows, Linux, Solaris, and Mac OS |
| Open Source | Open Source
and Free | Open Source
and Free | Open Source
and Free |
| Architecture | Headless Browser with event-driven architecture | Executes test cases directly inside the browser | Layered Architecture based on JSON Wire Protocol |
| Browsers Supported | Chromium, Firefox, and WebKit | Chrome, Firefox, and Edge | Chrome, Firefox, IE, Edge, Opera, Safari, and more |
| Support | Since Playwright is fairly new, the support from the community is limited as compared to Selenium | Strong community support from professionals across the world | Provides commercial support for its users via its sponsors in Selenium Ecosystem along with self-support documents. Strong community support from professionals across the world |
| Real Devices Support | Does not support real devices for Mobile Browser Tests but supports emulators | Supports real device clouds and remote servers | Supports real device clouds and remote servers |
- Do you use testing tools? Which ones? Marko: Yes, testing tools are a daily necessity for testing engineers. Some of the essential testing tools include FileZilla, Putty, Oxygen XML editor, Toad for Oracle, Testing Studio, HP ALM QC, Jira, and Selenium, as well as internally developed tools for testing team needs with the help of testing tools developers.
- How common is automated testing in practice? What tools do you use for that? Regarding user acceptance testing, how do you organize testing for more effective issue resolution? Marko: We heard the answer to this question during the presentation. Automated testing is most commonly applied in systems, system parts, and functionalities with relatively stable requirements, where the desired goal has been achieved, and a period of stagnation follows. Consequently, automation can be done extensively, primarily for regression testing and reporting to ensure that newly developed functionalities do not regressively affect existing ones. I would recommend using Selenium based on Python or Java, depending on further needs and applications.
- Here are five best practices for making the User Acceptance Testing (UAT) process efficient:
- Identifying the target audience: It is crucial to identify and understand the target audience, their problems, and their needs.
- Creating a testing plan: A testing plan outlines testing goals, scope, and a schedule of activities. It serves as a guide for testing efforts.
- Developing detailed test cases: Test cases specify the test procedure, expected results, and conditions to be checked. They provide a step-by-step guide for testers during testing.
- Establishing error communication standards: Clear guidelines on how to report and handle errors are essential for efficient issue resolution.
- Defining clear acceptance criteria: Clear and well-defined acceptance criteria are essential for successful UAT.
- Is there a demand for the SDET (Software Development Engineer in Test) role in Montenegro? Marko: I haven’t noticed a demand for this role in Montenegro. Even if there were, the question arises, as we discussed during the meeting, about how successful an individual would be in covering both testing and development in the long run. I believe that, in the long term, it may not be sustainable.
- Recommendations for materials for self-education in the field (for those with little experience)? Marko: Udemy offers a wide range of courses in QA fundamentals, manual testing, and automation. Since course prices drop to just a few euros during holiday periods, I consider them cost-effective for self-learning and gaining knowledge. You can also find courses related to Unix and databases, which I believe every tester should explore.
- How much programming is involved in the tester’s role in terms of writing automated tests, scripts, and more? Marko: To perform automated testing, you need to know Python or Java, at least to start with. Depending on the problem being automated, the level of programming can vary from low to very complex. Therefore, someone who decides to pursue automation should ideally be a good programmer. Often, automation tools provide templates, allowing you to reuse parts of the code with minimal modifications, but you still need to know when and where to use them and what needs modification.
- How common is End-to-End (E2E) testing, and what do you think of popular testing environments like Cypress and Selenium? Marko: End-to-end (E2E) Testing is a software testing method that verifies the entire software system from beginning to end, including its integration with external interfaces. The purpose of E2E testing is to test the entire software and its dependencies, data integrity, and communication with other systems, interfaces, and databases to simulate real-time production scenarios. E2E testing is typically performed after functional and system testing. Cypress and Selenium are open-source automation tools widely used for testing. I think they are suitable tools for someone starting with automation. For example, I would recommend learning Selenium with Python as a starting point. You can find suitable courses on Udemy.
- Are tester salaries competitive with developer salaries in the job market, and is there room for advancement? Marko: Tester salaries are generally lower than developer salaries. The reason is straightforward. Developers have a challenging job of building an entire system from scratch to full functionality, including performing modular integration tests. Consequently, there’s no need to explain further. However, the salaries of testing engineers who are professionals/experts can sometimes be in the same range as developer salaries, depending on the company and the tester’s contribution. There are companies dedicated solely to testing where rapid advancement in both position and salary is possible. Advancement opportunities always exist, and it depends on how quickly you can achieve it. My advice is that when you think you deserve a promotion, you must communicate it and fight for it!
- How do you comment on and execute tests, as well as record irregularities? Marko: Irregularities are recorded using professional tracking and monitoring tools for test execution, such as Jira or QC. These tools encompass the entire test issue lifecycle, from the moment they are reported to when they are resolved and closed. Each issue is reported with precisely defined rules and opened with specific comments/steps. Some of the elements that an issue must contain when reported include accurately described steps leading to the issue, a description of the testing environment, connection details related to Unix and databases, images of problematic system parts as evidence, and defined expected results, as well as actual results. The severity vs. priority of the error is also set. Ultimately, the issue is allocated through the system to the appropriate development department responsible for resolution. Tests are executed manually using testing tools.
- Whose responsibility is it to write tests, the client or the project executor, and who performs the testing? Marko: Writing tests and testing are the responsibilities of both parties but in different testing phases. In the unit, integration, and system testing phases, it’s the project executor’s duty and their testing team. In the User Acceptance Testing (UAT) phase, the client already has their test cases prepared and engages their testers to conduct system testing. Often, the project executor performs parallel UAT tests to proactively identify issues. Everything depends on mutual agreement between the client and the project executor in the end.
- Is it common practice to perform testing only until the beta version of the project, or is testing also allowed in production? Marko: Testing by the project executor must not be performed directly on the production environment. Working in the production environment is solely the client’s responsibility. If such work is necessary, an environment similar to production with copied data from the original production environment is prepared, and testing can be carried out on it by both the project executor and the client.
- Is it better to start as a developer and then become a tester, or vice versa (tester first, then developer)? Marko: We discussed this question during the meetup. Both transitions are possible, and the success of each depends on the individual’s adaptability. It is generally believed that a good tester can be a good developer because they know how to test and create test plans and understand the entire end-to-end system and its weaknesses and flaws. However, it should be understood that in the short term, this may yield good results, but in the long term, a tester who becomes a developer loses experience in testing and, consequently, testing quality. In the end, we would return to the initial situation. I lean more toward the rule that a tester should remain a tester, and a developer should remain a developer.
- How effective is it to write unit tests after the software development process solely to meet the code coverage standards of certain clients? Marko: Test-Driven Development (TDD) is a software development technique that involves frequent testing of the code being developed. Tests are written before the code, and then the actual code is written to pass the test. Refactoring is done to clean up and simplify the code further, but the fundamental principle is that a test once verified must be reverified every time there are changes. This is unit testing. Writing unit tests may make sense if it is defined in a contract between the two parties to meet specific code coverage requirements, even if it’s not practical. However, if a client demands it, we must write all unit tests to demonstrate that we followed the scheme, even if we didn’t do it in practice. These situations are avoided by writing unit tests in advance by developers who often don’t have time for such tasks due to their regular duties. The solution may involve getting help from testers to get involved and write unit tests instead of developers because testers should certainly know how to do it.
- Can you suggest some good tools for test automation (CI/CD)? Jovan: I would suggest Gitlab CI/CD, with which we at Coinis have had excellent experiences. You can add to this. Marko: I agree. Selenium with Python and Java has integration with GitLab.
- Will there be discussions on performance or security testing? Do your companies apply these, and to what extent? Marko: The meetup did not cover these specific testing topics because they were not the focus. Performance testing and security testing are often separate areas within the testing department. Security testing is particularly complex and belongs to a dedicated team within the testing department. These topics are discussed and conducted as per specific client requests, especially when the client does not have a security team and requires external testing. However, this may conflict with their security policies as they would need to open their security protocols to us, which can be challenging. Penetration testers often undergo special training and certification.
- Do you perform tests only for end-users, i.e., only what happens on the front end, or do you also perform tests on backend systems to identify issues more quickly? Marko: Each test case designed and executed includes steps to check both the front-end and back-end, along with operating systems and databases. All these checks are performed for each test case individually because each execution of a test in the front-end and back-end has a different impact, creates different logs, and records different values in the database. Consequently, they must be verified individually for each test case. End-to-end (E2E) testing is one of the testing phases in which comprehensive testing is performed for each test case individually, covering the entire functionality.
If you’re interested in taking a course in software testing, consider joining the Cortex Academy! Send an email with your personal information and your interest to contact@ictcortex.me, and we will be happy to provide you with free access to available online courses and assist you in furthering your knowledge in this field.
