How important is software testing and what’s involved?
Software testing is vital to the development of high-quality software, and the more crucial the system, the more important testing is. For example, transportation and medical devices are systems where the cost of error is extremely high, with human life at stake. These systems are always thoroughly tested, and should an error occur, the cause is investigated and documented to prevent future failures. In those instances, extensive testing and deep analysis of any failures are the only ways to ensure the safety of people using the technology. For the financial industry, software testing mitigates risk and prevents catastrophic financial losses and their grave impacts on the global economy.
In order to ensure the functionality of complex technology and manage risks, financial firms are increasingly focused on boosting resilience and software testing to provide necessary information on the resiliency of technology. For example, how the system reacts to different input data combinations or environment changes, and what exactly happens with the system when there’s an outage are key pieces of information when building a resilience strategy. The more scenarios tested through a system, the more coverage - and hence information – the process reveals.
Efficient execution of software testing requires an experienced team, a set of test automation tools and the right mindset to orchestrate the testing. The latter is underpinned by a number of principles. For one, software testing is relentless learning - testing means continuous improvement and involves research and constant self-evaluation against existing approaches.
Testing activities should start early in the software development life cycle and be carried out in parallel with development. Another principle is the deliberate practice of software testing, which suggests that not only should testing go through many iterations, but each time, the process should be improved.
What trends are you seeing from clients when testing software?
Various industries, including financial services, are increasingly adopting new technologies, causing a gradual conceptual shift in the perception of the inalienable characteristics of the existing highly complex technology platforms. For example, the application of artificial intelligence (AI) to financial technology indicated that existing technology systems also may demonstrate non-deterministic behaviour due to their extreme sophistication.
Similarly, with the introduction of distributed ledger technology (DLT) to the financial services industry, the systems became more decentralised, although distributed systems existed long before the blockchain era. In parallel, the adoption of cloud environments appears to have driven a better understanding of control over the test environments and the importance of data storage.
What are the biggest challenges in this space?
A key challenge is people underestimating the complexity of the technology that they operate, which often leads to underinvestment into test harnesses and monitoring solutions. We also frequently encounter challenges related to the box-ticking approach, when firms tend to run software testing only to satisfy compliance requirements, rather than to learn and improve systems.
What opportunities can software testing provide firms?
The core value of software testing activities is a comprehensive understanding of a system’s readiness for live service, information that is crucial in critical disruption scenarios. Building resilient platforms requires extensive non-functional testing to assess the system’s actual response to a major outage.
Those test results determine whether additional protection layers are needed, or whether monitoring mechanisms are satisfactory. For some firms, exploratory testing and process mining provide insights about the system’s expected behaviour.
When the specifications are absent for any reason, this description of the existing business flows is needed.
Looking to the next five years, how do you think software testing will advance?
We anticipate further progress of existing trends, particularly, the relationship between AI and testing. There are a couple of approaches that determine how testing and AI will continue to intersect.
First, there will be further exploration of the application of AI techniques to the process of software testing. AI methods are already being applied to intellectual system log analysis, failure prediction, process mining, defect classification and other areas. We expect both further advancements and increased adoption of AI into the practice of software testing.
The other approach uses AI as a target system for testing activities, an area where most breakthroughs are expected in the upcoming years.
To understand how these advancements might unfold, it’s important to understand that there is a substantial difference between software testing and proof of concept (POC). POC aims to prove that something works, while software testing aims to prove that something doesn’t work, to uncover the truth. Current AI-based solutions – for example, chatbots - are still at the POC stage and not production-ready.
We have spoken to some financial institutions who have integrated conversational assistants into their systems, and they report that chatbots are not ready – they are either biased, or their self-learning abilities are limited.
Even when AI systems seem to be ready, such as Microsoft’s Tay Bot and Google’s Meena, they have shut down due to bias-related concerns, rather than subjecting them to monitoring and deep research.
Chatbot challenges indicate how software testing of AI systems is crucial for the whole industry - it is better to test AI-based systems exhaustively rather than let those biases and bias-induced errors to be inherited by other, more critical AI systems, such as self-driving cars or medical devices.
← Previous interview
Brown Brothers Harriman
Derek Coyle
Next interview →
JTC
Nigel Le Quesne
Image: Shutterstock 
.jpg)