How the #1 Neobank Used Mobot to Eliminate Manual Testing

The Challenge

When you deal with people’s money, there is an extra level of scrutiny compared to other types of apps. Building trust with consumers is paramount, and nothing erodes trust faster than an app that doesn’t work.

This neobank’s app has a variety of complex features - new user account creation including sensitive bank account information, multi-factor authentication, check deposits, money transfers between devices, multi-device messaging to invite friends, and more. Many of these tests are very challenging to automate using traditional frameworks, and are not feasible using emulators or virtual devices. The customer has roughly 150 test cases that fall into this category, requiring manual testing from their team.

Adding to their already complex test cases, device and OS coverage are essential to ensure their millions of users have a seamless user experience. Building and maintaining the amount of physical devices needed for testing is costly and time-intensive.

To serve their ever growing user base, the customer releases to the app stores once per week with new products, upgrades, and general updates. In preparation for the weekly release, a US-based team member is designated as the person responsible for manually executing the test suite on the two physical devices in-house. On average, this manual testing would take 20+ hours each week.

• 12 million users are spread across a large array of device and OS combinations
• Has 150 test cases that are complex and very difficult to automate
• Manual testing for each release takes 20+ hours per week
• Number of physical devices available for testing is very limited

Mobot Solution

The neobank was introduced to the Mobot solution in 2021, while searching for ways to improve their release process. Their goals were to improve physical device coverage, eliminate manual testing for their internal team, and ensure they’re shipping the highest quality product to the app stores.

After a comprehensive onboarding consultation with the customer’s team, the Mobot CSM team designed and implemented a reliable and adaptable testing workflow. Using the customer’s existing test case checklist, the Mobot team built a suite of over 150 tests to be automated by the mechanical robots. 

To achieve the customer’s goal of increased device coverage, the teams worked together using the customer’s usage data to identify a rotating selection of 50 physical devices and operating systems that would be used in the tests. 

To ensure tight coordination and communication, a shared Slack channel was set up between the teams, and test results were integrated into the customer’s TestRails account.

The customer’s dedicated CSM became incredibly knowledgeable regarding their test cases and app, understanding when robot-reported results were bugs vs. design changes, and saving their engineering team countless hours sifting through data.

The Results

Today, the customer’s team can spend their time on higher-value work like exploratory testing, usability testing, and writing automation code. This new system where all critical manual testing is handled by Mobot’s human supervised, mechanical robots frees up over 20 hours per week of high value development time while also giving their team peace of mind.

Some highlights of the partnership:

• 150+ test cases automated
• Increased device coverage by 2,100%
• Saved 20+ hours of manual testing per week. Almost 3 full person-days of work
• Caught over 1,500 bugs before releasing to production