Software Testing: What I’ve learned so far

Everything that can be automated should be automated right? Nope! Wrong!! Well it depends as always. Let us try to dig a little bit deeper, or scratch the surface, or whatever idiom applies.

Modern software testing is, in my opinion, a tool to accelerate feature delivery to end users without breaking existing features in a seamless fashion. That should be one of the agile principles, if it is not then I strongly urge the powers that be to include it in.

Since I’m primarily involved in web applications, the following are a list of testing methodologies that I would use to test whatever web applications that I’m working on. It is presented in no particular order.

Real User Testing

This type of test exercises the software as closely as a real user would as possible. I like to call these UI Tests. This means the test should fire-up a browser and click around the web app as would a real user. This is usually done in some sort of pre-production or staging environment so that when the time comes to deploy stuff to production, the tests would have given the green-light to do so (or it could also give a red-flag and send the changes back to be worked on again).

These kinds of tests are more prone to become flaky and are a bit more time consuming than most other test types to run. So my mantra is to have a small set of robust tests that exercises the primary function(s) of the web application.

Some of the tools that can be used to perform real-user tests are:

• Cypress
• Selenium

Integration Testing

These types of tests are used to verify how the various parts of your application talk with each other. What I usually prefer to have here are API Tests that Create, Read, Update or Delete (CRUD) data. I believe it is enough to exercise the application’s REST endpoints to figure out that the system as a whole is working as expected.

It is advisable to have more of this type of test than UI Tests as they are quicker and cheaper on the time/resources scale.

As these tests generally perform HTTP requests, any library that supports HTTP calls such as GET, POST, etc. should be good enough to build a test suite around. Postman is another good alternative where we can design HTTP requests and manually test them. We’ll have options to later run them in Continuous Integration (CI) environments.

Unit Testing

This type of test is usually written by developers to drive their development processes (at least that is the principle of Test Driven Development aka TDD). These are tests that verify whether the code works as they were intended. The general consensus is that: write a failing test against the, often non-existent, piece of code, class, object, or function; run the test and watch it fail; make the smallest change that would pass the test; rinse and repeat. Aptly named as Red-Green-Refactor, this is a good approach to have a SOLID codebase.

There can be thousands or more of these tests as they generally do not use much resources and are quite fast to run.

Most, if not all, of the popular programming languages have libraries supporting unit testing. Just search for <insert language> unit testing, e.g. "php unit testing"

Other types of testing

• Synthetic Monitoring (with tools like Datadog or NewRelic) where a simple (but critical) piece of application workflow is verified on a predefined frequency. These can also be used as a heart-beat monitor where one critical endpoint is constantly monitored for its up-time. It is good to have them monitor the production environment to mitigate incidents before it reaches your customers.
• Performance Testing is yet another form of exercise for software applications. It will verify whether the app is good enough to handle a huge surge in user visits. It is generally used to answer questions such as “What would happen to our e-commerce store if there were tens of thousands of users shopping during the Cyber-Monday sale?”. There are many types of load testing which k6.io has detailed in-depth here. I have some good experiences using jMeter, Locust, k6.io but there are more tools out there

Why write tests at all

But why do we have to write tests at all? Most people find it a chore to write, hook-it-up in CI and maintain these (I know because this is a blog, a software on its own regard, but doesn’t have any tests backing it up). It is demotivating to have tests red-flag the feature that you diligently worked to create, right? Granted, but there are far more advantages to having tests.

There are entire departments in most decent sized companies managing this facet of software development. It is better to catch bugs in the system early in the delivery process than to wait for users to report it to you. Nobody likes losing customers and endanger their business as a whole.

Tests provide easy continuous integration and confident delivery. Fail early and fast. Get immediate feedback on potential production incidents and deliver a robust system in the end. These kind of chores are somewhat set-and-forget kind of a deal. Once automated processes are in place the rest is just how fast can you churn out features.

Cheers!