While multi-cloud strategies are complex to manage and test, Exactpro’s Iosif Itkin says employing the right mix of expertise and tools can ensure firms see the benefits of enhanced resilience, data governance and compliance while managing and eliminating the strategies’ downside risks
The cloud is officially mainstream.
According to the Flexera 2020 State of the Cloud Report, more than half of respondents have reached advanced cloud maturity level, with multi-cloud revealed as the dominant strategy, adopted by nearly all of the 750 organisations surveyed. Given the variety of benefits offered by a multi-cloud strategy, this growth is unsurprising, especially within financial services, an industry well-placed to reap the unique benefits the approach delivers.
Resilience and compliance
Combining resources from multiple clouds increases reliability – should one provider fail, firms can shift resources to another cloud provider, an especially attractive option in financial services, which depends on transaction processing being fully operational. Additionally, the industry has increased its focus on resilience following the market volatility in March and April as a result of the COVID-19 pandemic. For many firms, this has accelerated their digital transformation as they strive to boost resiliency through automation, thereby decreasing the odds of any type of failure or disruption as a result of human intervention. As part of advancing their technology infrastructures, firms will consider a multi-cloud strategy, and given its inherent reliability benefits, the industry is likely to see multi-cloud usage continue to rise.
Beyond resilience, multi-cloud provides attractive compliance benefits to a highly regulated industry. Not only does the approach facilitate more flexible data governance, but an adaptable cloud architecture empowers firms to optimise different security and compliance benefits from a variety of vendors. This promotes best practices for firms while avoiding becoming locked into one vendor, for better or for worse.
The trade-off is complexity. The ability to customise and combine the strengths of several cloud providers results in a more complex cloud architecture, which is more challenging to manage and to test.
An unbreakable link
From a testing standpoint, the technology is essentially the same when using multiple clouds, as are the functional aspects, as in, the system requirements and capability assessments. The difference lies in the non-functional requirements, including performance, latency, failover capabilities, ability to operate under significant load and ability to operate during short load peaks. These non-functional requirements are linked to their environment, and because multi-cloud systems are hybrid, they must be tested as a whole, including business logic, data, gateways and protocols, and application programming interface (API) interactions with both upstream and downstream systems. The inability to separate these testing processes from the cloud technology environment presents a test design challenge as to whether each specific test and/or test suite is assessing the system or the underlying cloud infrastructure.
To ensure test projects are thorough and robust, quality assurance (QA) teams must validate and verify the entire system, end-to-end. However, in a multi-cloud environment, differences in security levels, network and access settings create challenges regarding comprehensive test execution performed holistically against system components within different clouds.
Different cloud services may work through proprietary data management APIs, which introduces the need for the testing tools to reconcile the data from components associated with different cloud providers. Additionally, some cloud frameworks allow external interactions with the infrastructure while others require the test automation framework to be placed within and operate through the internal connections. Finally, supported encryption standard creates another variable that must be considered. This high level of variability in requirements creates challenges when performing concurrent actions generally originated by one test automation source across multi-cloud components.
Regulatory challenges
As is often the case in financial services, when the popularity of a particular technology grows, so does the level of regulatory scrutiny. The Financial Conduct Authority (FCA) guidance for firms outsourcing to the cloud and other third-party IT points out that while cloud usage can deliver greater flexibility and enable innovation, it introduces risks, and stipulates that firms cannot delegate responsibility for these risks to the service provider. The guidelines cover topics such as risk management, data security and change management, as well as specify that firms should have an exit plan should they need to terminate their relationship with a provider. Further, the FCA specifies that “where multiple providers form part of an overall arrangement,” the guidelines should be applied across the arrangement as a whole.
From a testing standpoint, rules around data security limit the range of possible ways to process text execution data and extract valuable information needed for monitoring and forecasting tasks. Should regulators choose to impose geographic restrictions on multi-cloud use, it can affect the efficiency of internationally distributed QA team operations.
Cost and control
Firms may find a multi-cloud strategy more cost-effective because it allows them to customise services from a variety of vendors competing to offer better services at a lower price point. However, it is important to factor in the cost of high-volume automated testing (HiVAT) performed in the cloud. The test harnesses connected to the platforms in the cloud generate diverse data flows under load, subjecting the system to various intermittent problems, like race conditions and data inconsistencies that will not be obvious if we just run tests one by one.
HiVAT is extremely beneficial to financial firms because it uncovers defects in the test environment, as opposed to in production, when a service or solution is under load with many clients depending on its reliable performance. It also opens up the possibility of streamlining the processes not only for the cloud, but for internal development and testing as well. However, the bill from a cloud provider after the cloud-based test environment is strongly stressed with HiVAT can be hefty, and it’s a factor worth exploring before making any decisions regarding a multi-cloud strategy.
In addition to cost challenges, there are two key operational hurdles to overcome when testing a multi-cloud strategy: testing team readiness and onboarding. The testing specialists’ skillset needs to be highly versatile to ensure they understand the specifics of different cloud technologies and are able to operate infrastructure management solutions for multicloud, using different tools for different cloud services when necessary. Before the start of actual QA activities, the external team needs to undergo a long compliance check process to obtain access to the client’s cloud infrastructure. Geo-restriction policies put in place in the organisation’s cloud can result in a third-party QA services provider’s offshore team being denied that access.
Enterprises striving for multi-cloud adoption should understand that even for the most experienced external teams, it may not be possible to overcome the difficulties stemming from the delegation of control and responsibility. In multi-cloud environments, information security, regulatory compliance, and high availability of the infrastructure strongly depend on external parties.
Additionally, emulation of outages on different nodes of the cloud network, as well as their geographical distribution, are controlled by the cloud service provider, not by the client who is (ideally) interested in having their system tested to its limits.?
Firms must weigh the benefits of virtual environment diversity, storage, and scalability against the loss of complete control over the quality of the enterprise technology platform as a whole. There is a related, broader benefit – the adoption of cloud environments appears to have driven a better understanding of control over the test environments and the importance of data storage.
Exactpro’s way to address these challenges involves two key components: people and tools.
Software testers devising and implementing multi-cloud strategies should be expert at managing distributed components in different cloud environments. In addition, the best are those that work at the intersection of testing, programming, and data analysis.
Looking at tools for testing resilience in multi-cloud environments, Exactpro has built a test automation framework, th2, which addresses several challenges at once. The platform is cloud-ready and supports extensive load testing through multiple iterations so that the client does not have to balance thorough testing with the cost of test runs executed in third-party clouds. This is crucial when a complex system needs constant and regular testing under load. In addition, th2 is tailored for major defect mining activities. It serves as storage for all test execution data and is enhanced with built-in test coverage analytics and compliance reporting capabilities. With a focus on extensive data processing, th2 makes it possible to execute more tests and run them concurrently under load, allowing for data-driven and model-based testing at the point where functional and non-functional approaches converge. The platform consolidates the power of the entire Exactpro test tool suite in a single solution and integrates with a variety of widely adopted test tools and frameworks via its open interface.
In a complex, interdependent industry like financial services, the benefits of a multi-cloud strategy are clear to see. While these strategies are complex to manage and test, employing the right mix of expertise and tools can ensure firms see the benefits of enhanced resilience, data governance and compliance while managing and eliminating the strategies’ downside risks.
