Tips, strategies and best practices for DevOps testing

Fri, 11th Dec 2020

FYI, this story is more than a year old

In a world where a pandemic has prompted organisations across the globe to mobilise and change outlooks to maintain business continuity, when it comes to software technology, companies need to take steps to ensure their software does not fall behind.

And in the realm of software development, perhaps nothing is quite as crucial as the need to scale DevOps. According to a recent report from New Relic, DevOps plays a vital role in ensuring shareholder value is consistently delivered in 2020, and that a lack of a true DevOps culture was hindering development teams' ability to develop more perfect software.

DevOps has thus changed software testing for good — a few years ago, when DevOps began its ascent, it represented a significant change to when and even how testing occurs. Testing was called upon more often, frequently at the hands of developers and in a more automated fashion.

But now, after considerable adaptation, testers are generally well-embedded into the DevOps environment, with specific DevOps testing strategies now commonplace.

Especially now, in an unpredictable financial landscape, delivering high-quality software at an accelerated pace requires quality practices to be continuous, highly automated, and to occur throughout the software development lifecycle.

So how can your team get there, and where should you begin? Here are five best practices for optimising software testing in DevOps environments.

Address the disconnect between quality assurance and developers

DevOps has changed the traditional sole testing focus of bringing a product to market. Now, with continuous integration and with new goals of increasing efficiency, DevOps requires testers to embrace a larger role as a quality professional, and take responsibility for the entire development lifecycle.

Having testers as part of an integrated delivery team can ensure quality is embedded in DevOps processes. Additionally, given the need to resolve bugs faster, it's essential to include the feedback cycle in that measurement as well. And the best way to shorten the feedback cycle is to involve testers in production.

Iteratively adopt practices and tools for each DevOps stage

Rather than a rip-and-replace approach to tooling, it's possible – and even preferred – to iteratively adopt practices and tools for each DevOps stage.

To begin, leaders should address the most significant problem areas first. This is a much more approachable method, as it allows teams to adopt tools and practices over time, as their processes mature and needs evolve.

According to recent research from Gartner, application leaders should drive DevOps teams to develop their practices and toolkit iteratively by identifying the biggest constraint in their processes and selecting the quality tools that best help them optimise for that constraint.

Application leaders engaging in procuring quality tools for their DevOps toolchains should align business objectives to the selection criteria to ensure the investment leads to business outcomes.

Adopt continuous testing practices

Organisations with successful DevOps initiatives have been found to make active efforts to transform software testing practices into continuous testing practices — more so than their peers, according to a Forrester study, commissioned by Tricentis.

These organisations are adopting five core continuous testing practices:

Allocating proper testing budgets and focusing on improving skills
Implementing continuous testing to meet the demands of release frequency
Including automating end-to-end functional testing
Integrating testers into cross-functional teams
Practising shift-left testing

According to the report, only 26% of firms with active DevOps adoptions are currently practising all five continuous testing practices.

Automate testing where it makes sense, but don't forsake test optimisation

According to recent research, successful DevOps shops are significantly more likely to have high levels of automation for critical testing and QA processes.

Test automation is vital in a fast-moving DevOps environment — but it's important to think about the bigger picture too.

It's crucial, according to Tricentis, to consider how to optimise the entire testing strategy, as test automation can't solve all manual process issues.

The goal should be to lean on the team's more tenured testers to develop a test automation strategy that maximises resources and eliminates the need for testers to perform repetitive test runs manually.

Efficiency should also be front-of-mind and can be achieved through finding the right mix of unit, integration and functional automation alongside manual and exploratory testing. To find this mix, testers must identify and report on areas of risk and take the time to review test plans, including test creation, test execution and amount of testing.

Focus on the right metrics

Determining the test cases that make the most sense to automate comes down to having good test data.

Centralising QA metrics to get a holistic view of all of the tests across all of the delivery pipelines will help teams see how frequently tests are failing, how long they take to run, and how much time is spent on environment setup and data creation.

Once a relatively high level of automation is achieved, the ability to quickly view and analyse metrics will become much more important. Access to the right metrics will provide insights that help focus test automation strategies on the areas in which it can have the most significant impact for businesses.