Let’s Talk About QA – I

After a few years working on different companies as developer and QA, I realized there are really low levels of quality, even understanding that sometimes there may be other priorities pretty usual to see huge shortcomings on quality.

For that reason, in a couple of posts, we will try to explain several fields of the modern “quality assurance”. It’s something common not to know what a QA Engineer does, it’s a relatively new position, usually mistaken with QA Tester position.

In a nutshell, QA Engineers are testers with development skills, but not just that. Also they should have knowledge about systems and DevOps to continuous integration and work really close to developers. Social abilities would be a plus, maybe you will need to work with non-technical professionals like product owners or project leaders to define which level of quality you will implement on the project.

Some basic concepts

Before talking about testing methodologies I want to refer you this link, where you can find a few slides with a dictionary of used terms in these articles. After that, you will be ready to know that there are different testing methodologies, born on traditional waterfall model, but they are useful yet.

  • White box: test the internal perspective of the component. Knowing how it’s built we can define which is the expected output.
  • Black box: completely opposed to white box, we find black box testing. It is completely oriented to exam the functionality of our application.
  • Grey box: grey box was born from the need to search for defects independently of structure or usage. This helps to find into the code where a functionality fails.

Other methodologies are oriented to agile frameworks:

  • Test Driven Development: it brings testing to the beginning of the development cycle, tests are defined and developed even before implementing the functionalities.

At this point we should do an important job of analysis, we should know how to do our job before it can run. This is really useful to improve the quality of a project, but it’s hard to achieve on young teams and/or on juniors positions.

  • Behaviour Driven Development: it has been developed as response to the problems applying TDD.

BDD provides shared tools and process to development and management teams, making easy to collaborate. This is facilitated through the use of a domain-specific language (Gherkins, for example).

Types of software testing

There are many kinds of tests, which cover different needs. Following a few steps you will know every gear of your project.

  • Design a test plan, each module of the project must be studied to identify each behaviour to handle and control.
  • Identify each area to test automatically on each module.
  • Each area will need different tools to automate.
  • Evaluate the results of each execution.
  • Identify possible issues of each iteration.

To know which kind of test we should apply we will study some of the most common types of tests.

Unit Testing: it is usually performed by software developers. It is a white box testing approach, where a developer has to test each component, with a complete review of each statement and assuring all the logic of the component is covered.

To test this component in an isolated way we should simulate the dependencies to avoid external interferences. We have some doubles to help us to do this job.

  • Dummy: objects are passed around but never actually used. They are just used to fill parameter lists usually and can be initialized to null.
  • Fake: object with working implementation but with shortcuts not suitable for production.
  • Stub: designed to respond a specific answer ignoring any other non-programed call.
  • Mock: objects pre-programmed with expectations which form a specification of the calls they are expected to receive. They are specially designed to test behaviour in contrast other doubles, usually used to test state.
  • Spy: variation of behaviour verification, Spy records calls made, usually ignoring any other information passed.

At Datio we use different frameworks to do unit testing, JUnit and TestNG for Java and ScalaTest for Scala.

Integration Testing: once individual units are tested by developers on unit testing, test team will design and run test to test the connectivity among those components.

For example, call a module to run a process which uses a complete set of functions. Each one must be tested alone on unit testing, but call a module and assert the complete result will prove all connections and definitions are right.

We use docker as ephemeral environment to do integration tests, with a docker-compose per project you can define a complete stateless environment for testing.

System Testing: one step beyond integration test we have to test the entire system due requirements. Same way you test connection inside a module, this, check connection between modules.

End-to-End Testing: similar to system testing, it involves testing of a complete application but mimic a real user interacting with it.

Functional Testing: test the application from a design document. Reproduce user’s steps and check the output as expected the user.

Regression Testing: test specially designed to find defects introduced with new commits, adding new functionalities or solving bugs. Great candidate to automation.

Acceptance Testing: this test is done to verify if the whole system meets customer specified requirements, fulfilling the acceptance criteria. Client do this testing to accept the application.

There are a lot of different kinds of tests. Usability, benchmark, load, stress, security… Not all kinds are responsibility of QA Team but you should know, at least, they exist.

Tools for testing

There are a lot of tools and tons of tutorials on Internet, here I would like to show you a few different tools and how they complement your unit and integration tests.

Test Management Tools:

First of all we need somewhere to document and control every functionality, this way you will have all the information to test your modules in the same place.

The principal benefits of this tools are clear on our daily work. It helps us to create and maintain project cycle information, specific tests and requirements per release, it gives us traceability and coverage, helps us to document each test execution and generates metrics and reports of it.

Probably, the most known Management Tool is Testlink (http://testlink.org/). It’s a bit old and have a non-intuitive interface, but, maintained by the community have almost every functionality you could need. It also provides (as many others) an API to work with it, one of the most useful uses is the auto-report of our test.

There are other powerful management tools, like Zephyr (from Jira creators), qTest one of the most valued tolls actually, HP Quality Center which it’s well known on business environments.

Static Analysis:

Each language used to have their own analyzer and you have to run different process to get multi language project information. Since SonarQube begins, we have a complete tool to analyze and measure technical quality.

SonarQube is specially useful on continuous integration, you can analyze a multi-language and multi-module project quickly, with your own rules and metrics and maintain a complete historic of results.

It has two well differenced components, a server side with a database to store all the rules of each language, all the projects and their evolution, as well every user and configurations. SonarQube allows to add a lot of plugins to obtain and improve functionalities.

There are several clients or analyzers, each one designed for different projects and languages, all do almost the same, read a config file from the project and send all the information to the server, where they extract all the information that you want through plugins.


Probably the most known tool for automation. It’s designed to automate browsers, you can reproduce every step in a web interface. It has implementations on almost every programming languages and works with the most used browsers.

You can define programmatically which steps should be followed to achieve a task and it will repeat each execution, exactly the same steps.

It’s a must for every web application to warranty all the functional paths are working for the users.


In the same way Selenium automates web browsers, Appium is a cross-platform tool to automate mobile apps. It works on simulators, emulators and real devices for iOS, Android, Windows and FirefoxOS.

It works thanks WebDriver protocol, the same used in Selenium. You define every step programmatically and run in the desired environment, is specially useful for multi-platform apps, where you write tests once and run on different systems.

Gatling and JMeter:

Quick mention to these two tools, both cover the same need, do performance tests, with different approaches, checking which load can handle your system automatically. We will do an article specifically about this kind of testing and how is implemented on our projects.

Beyond the terms

At this point we have enough knowledge to call each element by their name. This is extremely useful, having a clear image of our project and its structure we can cover most kind of elements without duplicate work.

On the next article I will talk about how to handle all these tests, not only with continuous integration and also with good practices. Implement all these in a company that use “different” methodologies is not easy. So we will talk also about that.

See you soon!


Jorge Barroso

I started to develop more than 10 years ago. Dedicating myself to the world of QA was a natural evolution to ensure that every project I enlisted in was the best expression of myself. Now I work as QA engineer in Datio.

More Posts